P2P:Protocol:Specifications:BitTorrent:External Sourcing

From Depthstrike Entertainment
Jump to: navigation, search

Preamble

  • This is a proposed extension to the .torrent metafile for torrents to add permanent seeds to a file that run on protocols other than BitTorrent.
  • The primary purpose for this extension is for content providers who wish to have permanent sources for their files without the need for dedicated torrent client software. Please make sure you either have a server capable of handling high amounts of http traffic or have implemented server-side controls to help limit bandwidth and access, as some client developers may attempt abusive accessing.
  • Please note that this proposal only has a partial demonstration implementation available as of this writing (March 28, 2006, Shareaza offline code. Code was imported into SourceForge CVS for Shareaza on April 7, 2006 and added to SVN in May.) and is purely experimental.

Metadata Extensions

sources

List of known sources for a given torrent. Added at torrent's root dictionary (to prevent issues adding/changing source lists)

Single-file Torrent structure

The "sources" entry in the root of the torrent is a list of sources for the file.

Multi-file Torrent structure

  • The "sources" entry in the root of the torrent is a dictionary of files within the torrent (nested folders are nested dictionaries).
  • Each file entry in the sources dictionary tree is a list of sources for that given file.
  • If a file has no http sources, it should be left out of the dictionary completely.
  • This layout permits sources for individual files within a batch to be hosted on different servers.
  • A special ":globalsources:" list in the root of the "sources" dictionary is used for the base URIs of these files. When using the global sources list, great care must be taken that file and folder names within the torrent are URI-Friendly.
  • A special ":flatsources:" list in the root of the "sources" dictionary is used for junctioning scripts and consolidated flatfiles. The scripts/flatfiles contain the contents of all of the files of the torrent, stored/referenced in the order listed in info\files.
    • When requesting from a script, the script will seek to the correct file(s) and upload the block/piece.

General Notes

  • Each value in a sources list is a URI pointing to the exact file (or an access-regulating script that relays the file)
  • These URIs need not be restricted to traditional HTTP sources.
  • Example non-http(s) uri prefixes:
    • ftp://
    • ed2k://
    • magnet:?
  • Not all uris are required to be supported, even if the client normally already support the uri format.

info/sourceequal

A number value of 0 or non-0

  • non-0 - Peers preference external sources equally to BT clients.
  • 0 - Peers preference BT clients over external sources. This is the default behavior if the value is not present.

Example Layout

  • PHP array style

Single-file

$torrentroot['sources'] = array( source1, source2, source3, ... , source<n> );

Multi-File

$torrentroot['sources'][':globalsources:'] = array( globalsource1, ... , globalsource<n> );
$torrentroot['sources'][<filename1>] = array( file1source1, ..., file1source<n> );
  • Each source in globalsources is a string with the uri/urn which represents the root of the structure of the torrent.
    • Files and paths in the torrent need to be replicated exactly on the server
  • Each source for each individual file ( <filename<n>> ) is a direct uri/urn to the file.
  • No example metafiles for multi-file torrents are currently available.

Consider external sources to be equal to peers rather than below

  • Enable external sources to be considered equal to peers:
$torrentroot['info']['sourceequal'] = 1;
  • Enable external sources to be considered lower priority to peers (default behavior):
$torrentroot['info']['sourceequal'] = 0;

Implementation notes

  • When dealing with multiple successive pieces, keepalives should be used. If the server doesn't support keepalives, the client should wait at least 5 minutes before retrying (unless told otherwise by the server, please see advanced implementation notes for details)
  • When deailing with connection errors, exponential growth on retry intervals should be used, starting at 5 minutes.
  • Unrecognized or unsupported uri/urns should be ignored permanently after discovering they're unsupported.

Advanced Implementation notes

Advanced Implementation notes