IceShare

restored, unedited from feb 2004

What is it?

IceShare is library that distributes Ogg streams on a pseudo-P2P network. It is heavily based on BitTorrent, but works on the Ogg page level, and unlike PeerCast it works with files as well as continuous streams.

It's designed to allow musicians, video producers, radio and television stations, or anyone looking to inexpensivly distribute audio/video on the web. It's intended to be initiated from websites, with links to icet:// URLs. It is not designed for P2P searching, such as Gnutella, Kazaa, and eDonkey provide, however websites may be setup to easily search content on one or more IceTracker servers.

Overview

IceShare is called pseudo-P2P because the network relies on a traditional client-server model for managing transfers between IceShare peers on the network.

The media players are the level which P2P takes place, whereas listeners who have available upstream bandwidth can help distribute the same content they're listening to amoung other listeners. This helps Icecast servers non-linearly scale for much larger listener loads and reduces the bandwidth requirements for hosting static Ogg multimedia on websites.

The IceShare library allows these features to be easily added to media players, including support for seeking to "not downloaded yet" parts of the media and available bandwidth detection/reporting for multi-bitrate streams.

IceTracker is a server that keeps track of who's activly sharing certain media and each of their send/receive ratios. IceTracker helps direct IceShare users to better hosts and track individual user's bandwidth and level of participation to reward high bandwidth/participation users with faster peers. IceTracker servers track users anonymously by a DSA key generated by each IceShare client.

Icecast connects to an IceTracker as a client to provide live stream information (pageno's, checksums, etc) and to receive guidance as per dropping less participating listeners when bandwidth is tight.

Media Players

URLs in the form icet://<icetracker>:<port>/<media> direct the media player to connect to an IceTracker using IceT protocol via the IceShare library. IceShare will state that it need's the specified media.

The IceTracker for that media should then respond with general information about the media in question, how many pages it has, how long its playtime is (or if it is continuous), and generally how long it should take to transfer it. This information should allow the media player to setup the seek bar and know how much it should buffer before beginning play.

IceTracker should then start directing IceShare to hosts which pieces of the media can be accessed from. IceShare does not know how much of the media each of those hosts has, since many may have only partial transfers. IceTracker specifies which page, or set of pages, to download from each host. IceShare responds with a quick "I got it" for each page, thereby letting IceTracker know that the reported page is ready to be shared with others. This also helps IceTracker keep track of latency and bandwidth between peers so that it can provide the client with better hosts.

If the player seeks to an not-yet-downloaded part of the media IceShare can express this to IceTracker, which will change its transfer focus to the seek point and beyond. In this way, especially for long pieces of media, the whole file does not have to be transfered to access a specific section of it.

IceShare also provides media players access to its "page table". The media player can use this to reflect media transfer stats in the seek bar, prehaps using an alternative background color to indicate sections of the media which have been downloaded.

IceShare handles incomming <A HREF="IceHTTP">HTTP</A> connections from peers, information about uploads on the P2P network are available to the media player but are not nessesary. The media player can tune the level of participation, limiting the amount of bandwidth or length of time a piece of media is available. For the most part, it's in the user's interest to participate as much as they're able to, since this will earn them faster access to other media through the same IceTracker.

A slightly-extended <A HREF="IceHTTP">HTTP/1.1</A> is used to specify page-ranges. IceShare should also support byte-ranges for traditional HTTP download agents which are attempting to resume a lost transfer.

Media Distributors

IceShare can also be used to distribute original media on the P2P network. A distribution client can use IceShare to connect to an IceTracker and inform it of the new media's statistics. This client should have enough upstream bandwidth to send the first few copies by itself, after which those who have downloaded it should begin sharing the load.

Icecast is a good example of a distribution client. It can use IceShare to inform IceTracker of its streams and continue to send it page information for each of its ongoing streams. Icecast servers using IceShare will still need enough bandwidth to send atleast one (preferably more) streams to listeners who can then redistribute it to other listeners.

IceTracker will allow IceShare clients to request current listeners and total "hits" for any media that it is tracking. This can be used by Icecast to accurately track listeners.

Alternative Streams

IceShare also includes support for alternative bitrates and codecs to published media. These alternatives can be used to meet the needs of each individual user on the network. For instance, a stream could be provided in 64kbps Vorbis, 24kbps Speex, and 24kbps Vorbis (in that order). Those with enough bandwidth will receive the default 64kbps Vorbis stream, while modem users will switch to either the Speex or the low bitrate Vorbis based on their ability to support Speex. This makes it possible for every IceShare peer to receive a continuous stream in the highest quality format their software and network connection allows them.

New Website

The new project website for IceShare is http://www.iceshare.org/ ! J_Bullet has volunteered to design the site, but it's just general information for now.

What's the Holdup?

While we're apparently fairly close to wrapping up this baby and start into some massive plugin coding, there's vital things missing from other Xiph projects which this needs. Specifically, OggFile granule handling needs to be decided on so we can get the IceT timing specs to match, we need to get the final scoop on discontinuous bitstreams, and a fully functional public tracker needs to be written (even if it's just in Python for now). Plus, it makes alot of sense for the plugins to be packaged with OggFile as a join distribution effort. Current estimates, given the committments from various Xiph developers and pace of development, is IceShare will begin getting deployed summer 2004. Plus, as revolutionary as it is, it'll take a few months for it's useage to be widespread enough to useful for general usage until atleast the end of 2004.

If you're reading this page and thinking "damned, that's awesome!" and want to speed up the above timeline we can always use volunteers. Even if you don't know any code, but prehaps you're good at HTML or graphic design, or can do no more than help test the system, we could use your help. Email Arc Riley <arc@xiph.org> for more information.

Discussion

... --2003/11/21 13:36 EST
yo xiph: quit doing so much cool stuff, you make me feel worthless.

Killer app? --2003/11/22 14:06 EST
I can't see any competition :D

re: Killer app? --2003/11/28 18:39 EST
take a look at peercast http://www.peercast.org/ (be shure to disable ECN ;) ) it's working already!

keep the work going!

Wow, wow, wow! --2003/12/01 21:04 EST
Wow, I was talking to my Bro about this idea a few months ago, and how it could change the world. I was talking about video, but Audio is a good start!

It will do video. --ArcRiley, 2003/12/10 01:52 EST
IceShare will work for any Ogg codec, including Theora. In fact that's one of the uses I intend to test it with.

hmm --BlindWanderer, 2004/01/05 10:47 EST
Swarming streaming would work but i don't think it would catch on. But it still should be made. It would help guarantee freedom of speech/press in this world of censorship.

... --2004/01/12 01:33 EST
Video might end up being the killer app. Hosting an audio stream is expensive, it is still feasible. Hosting a video stream, on the other hand, rapidly becomes unaffordable. This project could make it affordable to broadcast high-quality video streams to a large audience

Where's the download link? --2004/01/19 22:35 EST
?????????????????????

Work in progress --ArcRiley, 2004/02/04 08:25 EST
This is currently in the "pre-alpha experimental and design" phase. We have some software, mostly for testing, and the protocols are still being altered slightly or extended as we need to. Don't worry, we'll make a big media splash if/when we get to beta release phase.

... --2004/01/21 22:38 EST
will it be posible to listen to a radio with winamp?, i mean no need for a special player? how coul we broadcast? using a special program?

Winamp --ArcRiley, 2004/02/04 08:27 EST
Short answer; when someone writes the software for this, and then only after we've finalised the spec. WinAMP will need a plugin, but that is all. The intention of IceShare is for it to be integrated closely with the media player, so not only should icet:// links open your media player directly, but things such as seeking should work seamlessly. As for broadcasting software, that's left completely up to implementation. Icecast support should be written eventually so any standard Icecast2 server can be used, but anyone could write a direct IceShare streaming client. Let's focus less on vaporware, right now we're concentrating on the spec, not integration with media players.

local relay --Andy Baxter, Sept 2004
Seeing as it might take a while for plugins to be written for all the players, how about making a local relay program for the main platforms which uses the iceshare library to provide a local http or rtsp relay, which players can connect to? This would also do as a reference implementation of the library for plugin coders. If the right protocol was used, it could handle seeking etc as well.

misinformation --2004/03/01 11:17 EST
I think you may have been misinformed about PeerCast, we're using it as the core of our media player and are streaming static files with it. Security was a concern for us so we added a very simple SKEY signature to broadcast packets that allows clients to verify each packet has not been modified. Although we haven't submitted patches to the PeerCast developers yet. Also our client is "allowed" to connect o to the main PeerCast network, in fact the developers have been very eager for us to do that.

Conferencing --AlexanderWinston, 2004-03-03 00:00 UTC
Might this have possible tele- or videoconferencing uses?

Dead Project --2004/11/09
Was a great idea, too bad it never took off.

Don't Assume -- 2004/11/19
The wiki just hasn't been maintained, work is happening behind the scenes. Be patient.

2004/11/19 I am very glad to hear this news! I am looking for something just like this & this looks very promising. I have started an internet radio station and am looking for a way to expand the stream once the audience exceeds my server capacity.

I know nothing of writing code but could beta test or write a "how to" article.