- 1 Concepts, Examples and Questions
- 1.1 How does the tracker link itself work? How does it share peer information?
- 1.2 I have an existing tracker that is having problems, how do I link it to another to gain additional performance?
- 1.3 How does linking multiple trackers balance the load?
- 1.4 What steps should we take to get the link to work right?
- 1.5 One or more of the trackers on my network was disconnected from the network for an extended period.
- 2 Configuration
- 3 Proper Torrent Generation
Concepts, Examples and Questions
The tracker link works and shares peer information in much the same way that multiple-server IRC (Internet Relay Chat) networks relay chat information. As a peer announces to one tracker, it's announce request is processed and sent across the link to the hub and other leaf trackers and stored in the individual tracker's peer database. If the trackers are allowed directory enabled, and an announce comes in on a tracker over the link for a torrent that the tracker doesn't have, it is ignored and removed from the update queue. On torrents that are common to multiple trackers, all trackers that have the torrent act as if they were one tracker.
Four trackers, Trackers 1-3 and Tracker-hub
A client announces to a torrent common to all 4 trackers to tracker 1.
The information about that client (peer id, IP address, port, torrent infohash) is relayed to the tracker hub, where it is then relayed to the remaining two trackers.
A client announces to tracker 2 on the same torrent. It gets the peer information given to the network by the client that announced to tracker 1.
This is where just linking a new tracker and hoping the load distributes automatically won't happen. You need to modify all your existing torrents for use on the new tracker network or configure your tracker's DNS to point to all trackers at once. If you have trackers running on different ports for each system in the network, or your DNS registration does not permit round-robin DNS, the DNS option just plain won't work.
Also, existing downloaders will need to re-download the torrents in order to take advantage of the new tracker network if you can't set up the round-robin DNS. New downloaders won't need to make that change.
The best way to adjust a lot of torrents at once for multitracker is to use the btreannounce.py from the BitTornado Source Package with Python installed. Once python is installed and the BitTornado source package is extracted (I extract mine so that btreannounce.py is in c:\btsrc) you can use the following command on windows from your allowed directory or torrent hosting folder (or a similar command on linux, i don't know what it is) to edit all the torrents at once.
for %a in (*.torrent) do c:\btsrc\btreannounce.py (announce URL) --announce_list (list of announce URLs for each tracker, all comma separated) %a
What this command does is goes through and automatically changes the announce URL in all the torrents in the folder to match the one you specify and generates the announce-list for the file in a proper manner so that supported clients will automatically load balance.
The BitTorrent tracker code is currently very optimized, using almost immeasurably small amounts of bandwidth even for thousands of peers, so bandwidth concerns usually aren't all that major even in single tracker environments. When you get problems that require multiple trackers to be linked you are dealing with issues relating to numbers of connections not bandwidth. What linking multiple trackers does is spread those connections across multiple systems that can't handle much on their own, but as a group can handle as much if not more traffic of their bigger single-tracker cousins.
Adding a tracker link does not automatically balance the load, as I have described above, but will allow for growth in quite measurable ways in the future. A friend and I are both running trackers and want to link to each other.
The steps you should take are:
- Make sure you're both running link-capable tracker software. Currently only BNBT and those built on it (BNBT official versions 7.3, 7.7 and 8.0, BNBT Trinity Edition 7.7, and CBTT 7.3, 7.7 and 8.0, as well as some others as of this writing) are link capable.
- Make sure that if you're both using allowed_dir enabled trackers that you get the torrents from each other that are missing from each tracker.
- Make sure that you have the link established properly by double-checking the administration panel of the tracker that you made into the network hub (yes you can make a dedicated tracker hub separate from your main trackers)
- Make sure that you have modified all the publicly available versions of the torrents that you are sharing the load for to include the announce-list for your fresh network.
- Make sure to set up a mechanism for the trackers to automatically share torrents between them. Guidelines for Syncronization
One or more of the trackers on my network was disconnected from the network for an extended period.
What special steps do I need to take if this happens?
If the trackers re-establish their connections properly on their own, you do not need to take any special steps to resynchronize them unless you released new torrents while they were down. If you released new torrents, you need to add them to the trackers that were down manually. Other than that, the tracker link will resynchronize the affected trackers automatically over the span of 1 hour and 30 minutes (3 announce cycles, 2 ghost peer expiration cycles)
Sometimes a tracker does what is called a "soft-split" where the link is still established and listed as active in the administration panel, but no peer information is being sent over it. If that happens, you need to go to the administration panel of the affected tracker and choose the "Reset Tracker Link" function to kill the connection to the hub and re-connect it. After that, resynchronization happens normally after 1h30.
The IP address that tracker link communication is handled on. Only use this when you KNOW you need to, since incorrect bindings will prevent the link from working.
The IP address or domain name of the tracker network hub you want your tracker to connect to. This is not to be used by the tracker network hub. Because this can be a domain name, the tracker hub CAN be on a dynamic domain name.
The password to be used by the tracker network. This must match across ALL trackers on the network.
The TCP port used by the tracker network. This must match across ALL trackers on the network. The tracker hub must be configured to allow inbound communication on this port. Running firwewalls that allow you to restrict access may help improve network stability. Allow only communication from IP addresses that you know are going to be hosting trackers for you. - Valid range: 0-65535, recommended range: 1024-65535. One port from the range.
Is this tracker the hub of the network? (0 - no/1 - yes)
for a tracker hub
(address: tracker-hub.testsite.com) bnbt_tlink_bind =
bnbt_tlink_password = bnbtLinkTest
bnbt_tlink_port = 59100
bnbt_tlink_server = 1
Example configuration for tracker leaves to the above hub
addresses: tracker-x.testsite.com where X is a tracker number):
bnbt_tlink_connect = tracker-hub.testsite.com
bnbt_tlink_password = bnbtLinkTest
bnbt_tlink_port = 59100
bnbt_tlink_server = 0
Notice that the bind values are blank for both example configurations. This is because we don't need to bind to a specific IP address at all unless we have computers with multiple network adapters and we want the link to operate on only one of them (example, running a link over a 100mbit backbone while not accepting link requests from the internet).
Proper Torrent Generation
In order to properly take advantage of the tracker link for load balancing, torrents need to be made in a specific manner. BNBT's tracker links properly support being used in TheShad0w's Multitracker Specification Local Mirror
Only specific metafile generators can properly make torrents for a multitracker network made with BNBT. These include the built in torrent generator in ABC, the btmaketorrentgui.py in the BitTornado source package and the integrated torrent maker in Azureus. MakeTorrent only supports one of the three methods of torrent layout described in the multitracker specification, and does not perform proper load balancing. Because of this flaw, MakeTorrent is NOT recommended for use in generating torrents for multiple tracker networks.
Torrent layout (ABC torrent maker/BitTornado btmaketorrentgui.py):
- http://tracker-1:2834/announce http://tracker-2:2834/announce http://tracker-3:2834/announce
Please note that all the trackers need to be on one line (single tier in the example from the specification) to properly load balance across all trackers, and all peers need to be using a properly multitracker-enabled client such as ABC, BitTornado, or Azureus