Summer of Code 2007

Please see Summer of Code for the current project ideas page.

Ideas

This is our ideas page for Google Summer of Code projects with Xiph.org and Annodex. The two projects are participating jointly this year under Xiph's name.

We need a primary and backup mentor volunteer for any project that is to become an official proposal, but submit something and we'll see who we can round up. :)

note: Google Summer of Code 2007, mentoring organizations to apply between March 5 and March 12, students March 14 - March 26

Students please use the template at Summer of Code Applications when applying for a GSoC position.

Mentors for details of our mentor application and plan, please see Summer of Code 2007.

Students should also check out projects related to the Elphel Open Source cameras.

Optimize Theora encoding/decoding speed, SSE/SSE2

Work on MMX, SSE/SSE2 implementations of the crucial encoding and decoding elements in libtheora and/or theora-exp. This could include porting the vp3 mmx and altivec code to the libtheora decoder, and writing sse improvements on the mmx work that has already been done. The results must still build cleanly on other archs and do run-time capability detection.

Mentor: Ralph Giles, Timothy Terriberry, backup: Jan Gerber, Mike Smith

Encode support in theora-exp

Implement a rate-distortion optimized encoding mode for theora-exp, including R-D optimzed mode decision and quantization (e.g., constant lambda). Then, use the above routines to implement a medium-latency ABR encoding mode (e.g., varying lambda), with a default target buffer size of approximately 2 seconds.

Mentor: Timothy "Derf" Terriberry, backup: Mike Smith, Ralph Giles

Development assistant for the "Ghost" audio codec

Designing a cutting edge perceptual codec is a very daunting task. Xiph is in the research stage on a new low-latency, general purpose audio codec, code-named "Ghost". This is basically a "code assistant" position, where you will be asked to implement, test, and give feedback on ideas from Christopher Montgomery, designer of the Ogg Vorbis format. Be prepared to learn a lot about audio coding, or apply what you already know. While there's less "ownership" potential in this project proposal, it will be a great opportunity to learn about compression algorithm design, practice your programming chops, and learn to work in team.

Mentor: Christopher "Monty" Montgomery, backup: Jean-Marc Valin

OggMNG implementation

Implement the OggMNG decode support in gstreamer and/or illi's dshow filters. Implement encoding support in based on byzanz or Istanbul. Bonus points for overlay support. Details on the OggMNG specifications here

Mentors: Mike Smith, Ralph Giles

Theora reference encoder quality optimization

The libtheora encoder could make more use of some features present in the spec but not currently implemented in the encoder. This is a little open ended, but suggestions are: quant matrix tuning, per-block qi choice, 4:2:2 and 4:4:4 chroma support.

Mentor: Ralph Giles, backup: Timothy Terriberry

Subtitle Definition

There has been a long-standing need for the introduction of subtitles into Ogg. Several means have been suggested and various implementations exist. However, there has been no standard way that is supported by Xiph at this stage.

The CMML format with its time-aligned means of interleaving text into Ogg bitstreams is a platform on which we would very much like to define a standard means of including subtitles.

In this project, a standard means of interleaving subtitles (as found on DVDs or in srt files) into Ogg will be defined using CMML.

The project requires to make changes to the CMML definition and extend it in several ways. CMML needs to have a valid XML schema or DTD definition associated with it, so that standard XML tools will parse it. The associated documentation should then be updated and software written to put e.g. a srt file into CMML inside Ogg. If there is enough time, it would also be good to implement support for this format in a media player such as vlc or mplayer or xine.

Mentor: Silvia Pfeiffer, backup: Conrad Parker

Theora support in ekiga

Implement support for Theora as a video codec in the ekiga chat application. Overlaps with GNOME.

Mentor: Ralph Giles

MXF support in gstreamer

Implement an MXF mux/demux for gstreamer, with mappings for Vorbis and Theora.

Mentors: Christian Schaller, Mike Smith

Cascading Style Sheet support for CMML in GStreamer

Implement support for Cascading Style Sheets to add styling and positioning hints to CMML text overlays on Theora video. Doing so allows for advanced titling features visually similar to TV-style news headlines, sports scores, and scrolling text. The advantage over conventional "burnt-in" titling is that the stylesheet-driven approach is machine-readable, allowing indexing for search and improved accessibility.

The project involves:

1. YUV compositing support in the textoverlay plugin in gstreamer in order to complete the low-level support for color and font attributes. Textoverlay already includes partial support for pango text, but lacks the necessary colorspace conversions. This portion of the project necessarily involves C programming.
2. Implementation of a test application to playback video marked up with CSS titling hints. It is recommended that this portion of the project be implemented in a higher-level language for which GStreamer support and CSS parsing libraries exist, such as Python, Ruby or Haskell.

Further work could include support for style sheets in liboggplay / Firefox, or direct support for style sheet retrieval and rendering in GStreamer (via the cmmldec plugin and/or a new style-sheet-aware textoverlay plugin).

Mentor: Conrad Parker, backup: Mike Smith, Silvia Pfeiffer

Hardware implementation of Theora decoding

Working on a hardware theora decoder, that can be used in embedded devices, dvd players and video pods. Presumedly GPL verilog source to run on an FPGA. See http://sourceforge.net/projects/elphel/ for a rough encoder implementation and http://svn.xiph.org/trunk/theora-fpga/ for the decoder implementation. This was a successful project in 2006.

Intel to AT&T x86 assembly translation

There is a general need for cross platform projects to be able to compile the same asm accelleration code on both GCC and MSVC. Unfortunately, at least of x86, they have incompatible assembly formats. Currently people either convert one to the other by hand (a maintenance nightmare) or require an external compile/assemble step on one or the other platform.

Start with the (unmaintained?) intel2gas script. Spruce it up to support all of recent MMX, SSE, SSE2, SSE3 instructions. Then implement the reverse translation. Once both are working, write some glue code so it can be easily used as part of a GNU autotools build to derive one set of source from the other at build or package time.

Speex and FLAC encoders in Xiph QuickTime Components

Implement Speex and FLAC Core Audio encoders.

XiphQT has a Vorbis encoder component that could be used as a reference and starting point.

Mentor: Arek Korbik

New vocoder for Speex

This is a real speech coding project! Speex currently has a very low bit-rate (2.15 kbps) mode that is implemented as a trivial vocoder. This mode has four "reserved" bits per frame, which means it would be possible to transmit more information. The idea of this project would be to make use of these bits to improve the quality of the 2.15 kbps mode. Changes to both the encoder and the decoder are allowed, provided that they are compatible with older version. This means that the new bit-stream should be decodable by the old decoder with only minor loss in quality. This still leaves plenty of room for improvement. Upon successful completion, there will be a new vocoder that can be merged into the main Speex tree with improved quality and (non-exact) backward compatibility with the previous vocoder. The project requires some signal processing knowledge.

Mentor: Jean-Marc Valin

New Speex VAD/VBR code

The current Speex VAD/VBR code is a quick hack, put together a long time ago. This project consists of rewriting it to perform much better under all kinds of conditions. Performance criteria are: classification accuracy, quality in VAD/DTX mode, simplicity, and of course speed. This project involves pattern classification and speech processing, so it requires at least some knowledge in these fields (or at least one of them).

Mentor: Jean-Marc Valin

rehuff: a tool to losslessly compress Vorbis files

Would be nice to have an updated version of "rehuff", a tool to losslessly compress Vorbis files. There were an experimental version of it (see rehuff status), but had some limits:

• it's not free software;
• it has a bug causing the rehuffed file can't correctly seek;
• it works only with stereo files.

For more information, see also Rehuff.

Would be nice to have an updated rehuff, without the previous limits, and with a library part that will be included in libvorbisenc, so all encoders could use it (rehuff binary, oggenc, ...).

Not an official Xiph.org project, only a user proposed idea.

Ogg and Annodex integration into open source web Content Management Systems

The goal of this project is to make the use of ogg theora video in existing CMSs as easy as possible. The project would consist of integrating in browser playback and structured cmml metadata into existing CMSs like mediaWiki, drupal or wordpress.

In browser playback support will be handled by vlc or java cortado plugin and then liboggplay & native browser decoding as that project matures. mv_embed may be a starting point for client plugin detection. The extension package should also handle thumbnail generation via mplayer or ffmpeg, and ideally support transcodeing via shell calls to ffmpeg2theora. Meta data attributes for the video in the CMS could be exportable as CMML. (a standardized xml format for tagging continues video)

Mentor: Michael Dale, backup: Silvia Pfeiffer

OggPlay: Time Offset Acceleration

OggPlay is a new library that enables developers to drop ogg media support into applications. OggPlay will be used to implement native Ogg/Annodex support in Firefox, and supports a range of features including playing Oggs provided in TCP streams.

This project involves adding time offset optimisation support to OggPlay in TCP mode. Upon successful completion, applications will be able to notify the library of "interesting" time regions of the file, either at the current time point or in the past or future. The library in turn will ensure that these regions are pinned in local memory or on disk using a combination of compressed stream and uncompressed frames. If the application later attempts to seek to a pinned time region, then access to the stream at that point will be much faster than other regions.

Time offset acceleration will be useful to projects such as Annodex, which allows annotation of time regions ('clips') in Ogg, as well as direct access to individual clips over the WWW, URL-based naming of clips, and linking between clips.

Mentor: Shane Stephens, backups: Marcin Lubonski, Michael Martin

Guidelines for Applying

Remember that many people will apply to work on the Summer of Code.

Keep in mind that those of us evaluating your application do not know you, we do not know what kind of experience you have, we do not know what you have done in the past and we have to pick the best people suited for a particular task.

Hence, it is very important that you tell us in your email why you should be considered to implement a particular project. Please use the application template at Summer of Code Applications as a starting point.

See Also

• CodingGuidelines
• MIT approach to design and implementation
• How to do a release

Application

Google Summer of Code had a formal application process for mentoring applications. The window was March 5-12, 2007. This section was for development of our joint application with Annodex.

Here are the questions the GSoC faq said we must address:

Describe your organization

Xiph.org is an open source project and non-profit corporation dedicated to providing open and free-to-implement multimedia technology as a foundation for an interoperable, level playing field on the internet and other digital distribution networks. Over the past 8 years we have hosted development for all the major patent-free audio and video codec development, including the Vorbis, Speex, FLAC and Theora, the Ogg streaming format, and the icecast streaming media server.

This year we are also coordinating projects for the Annodex association under our umbrella. The Annodex project is developing a set of open specifications and open source software to allow the creation of hyperlinked Webs of audio and video integrated with the text-based view of the current Web. Toward this goal, Annodex has done a great deal of work developing tools, browswer plugins and convenience libraries to facilitate adoption of Xiph.org's lower-level technology. As such the two projects have largely aligned goals, but focus on different levels in the stack.

Why is your organization applying to participate in GSoC 2007? What do you hope to gain by participating?

We believe that Xiph.Org has a wide-ranging set of projects that are both challenging and educational for the students. Furthermore, they are important/useful goals for the wider technology and especially the open source community.

We believe that Xiph's mandate to develop multimedia standards and software is an important one, but we--of course--have limited resources. We hope that the results of GSoC will include the direct benefit of new software development, but also help grow the number of active participants in a long-term manner. Most of our core developers started as students, but just as many have moved on since their student days. Attracting and retaining students is essential to the health and sustainability of our project and is an important goal for all of us.

Did your organization participate in GSoC 2005 or 2006? If so, please summarize your involvement and the successes and failures of your student projects.

Xiph.org was invited to participate in GSoc 2006, and informally mentored annodex-related projects as well.

We were granted funding for 6 slots. One we weren't able to fill because our chosen students picked or were assigned to other projects.

Two were successful. One, a hardware implementation of a theora decoder: this project produced HDL implementation of the major decoder components, to be used with a general purpose CPU, such as the open source LEON sparc implementation. Real time playback of SD content was demonstrated using these components in combination with the proprietary Nagios cpu design. Two, implentation of OggSkeleton support in various tools. We have maintained contact with the two successful students since the program finished. One has continued to contribute code outside the GSoC term, continuing related work.

The remaining three were unsuccessful through lack of necessary skills, health complications, insufficient motivation, or some combination of all of these.

We enjoyed our participation last year. It energized our project and improved our connections to the rest of the open source community. We learned some lessons about mentoring and especially about applicant screening and would appreciate a chance to apply them in a second round.

If your organization has not previously participated in GSoC, have you applied in the past? If so, for what year(s)?

We did not apply outside of our invitation in 2006.

Who will your organization administrator be? Please include Google Account information.

Ralph Giles: google account giles@xiph.org

What license does your project use?

In general, we use a modified-BSD style license for our libraries, to enable the widest possible uses of our formats and reference implementations.

Our applications are generally GPL but LGPL or (GPL-compatible) modified BSD is also acceptable.

What is the URL for your ideas page?

This wiki page contains our current suggestions: (updates needed for 2007):

http://wiki.xiph.org/index.php/Summer_of_Code

What is the main development mailing list for your organization?

We don't have a single central mailing list for all of Xiph.Org. Instead, we have per-project mailing lists; of these the most active is vorbis-dev@xiph.org. A complete listing of our lists is available at:

http://lists.xiph.org/mailman/listinfo/

The primary contact of the Annodex projects is annodex-dev@lists.annodex.net

What is the main IRC channel for your organization?

We use freenode (irc.freenode.net), with a number of channels. #xiph is the main organisation-wide one, but we also have project channels: #vorbis, #theora, #icecast and #annodex are the major ones.

Does your organization have an application template you would like to see students use? If so, please provide it now.

http://wiki.xiph.org/index.php/Summer_of_Code_Applications

Who will be your backup organization administrator? Please include Google Account information.

Michael Smith: google account mlrsmith@gmail.com

Who will your mentors be? Please include Google Account Information.

Mike Smith: google account mlrsmith@gmail.com

Mike Smith has been involved with Xiph.Org and multimedia development since 2000 and remains one of our most involved core members. He helped write major portions of the vorbis-tools suite and the icecast streaming media server. He currently works as a developer at Fluendo.com on the GStreamer multimedia framework, flumotion (a streaming media server), and other multimedia software.

Ralph Giles: google account giles@xiph.org

Ralph Giles has been involved with Xiph.org and multimedia development since 2000. He has contributed to tool development and the theora video codec. He handles much of the coordination and adminstrative work for Xiph.org. He mentored the successful "Hardware implementation of Theora decoding" project in GSoC 2006 and acted as an administrator.

Silvia Pfeiffer: google account silviapfeiffer1@gmail.com

Silvia Pfeiffer is the founder of the Annodex project and president of the Annodex Association. She is the principle author of most of the annodex specifications, RFC 3533 and RFC 3534 describing the Ogg media format, and is heavily involved as an organizer in both the Xiph and Annodex projects. Most recently she organized the FOMS developer summit and media recording for LCA 2007. She helped mentor the successful "OggSkeleton support" project in GSoC 2006 and acted as an administrator during the application process.

Christopher Montgomery: google account xiphmont@gmail.com

Christopher "Monty" Montgomery is the founder of Xiph.org, architect and lead developer of the Ogg Vorbis general purpose audio compression format. He has been doing open source development under the Xiph name since 1999. His current work focusses on development of a next-generation audio codec. He helped mentor a project related to this in GSoC 2006.

Timothy Terriberry: google account tterribe@vt.edu

Timothy Terriberry is the author of the Theora video codec specification and the author of the theora-exp implementation. He has been an active contributor since 2003. He mentored a theora-related project in GSoC 2006.

Jan Gerber: google account jan.gerber@gmail.com

Jan is the author of the very widely used ffmpeg2theora transcoding tool, and maintainer of the v2v.cc syndication website. He has been a project contributor since 2002.

Conrad Parker: google account conrad.parker@gmail.com

Conrad Parker is the author of the liboggz and libfishsound convenience libraries and the sweep audio editor. He is heavily involved in the Annodex project and mentored the successful "OggSkeleton support" project in GSoC 2006.

Jean-Marc Valin: google account jmvalin@gmail.com

Jean-Marc Valin is the architect and lead developer of the Speex voice codec. He has been involved in Xiph.Org since 2002. He mentored an audio codec research project in GSoC 2006.

Arek Korbik: google account arkadini@gmail.com

Arek Korbik is the author of the Xiph QuickTime Components. He has been a project contributor since 2005.

What criteria did you use to select these individuals as mentors? Please be as specific as possible.

We selected our mentors from the 'core' developers and contributors within Xiph. Mentors were selected based on how well they know the code area they're volunteering to mentor, how long they've been part of Xiph, how well they interact with others (particularly in terms of building community around our projects).

The majority of the mentors we've selected are core developers on the various Xiph.Org sub-projects that they've volunteered to mentor for. They have been contributing to Xiph.Org for at least several years, and have shown a persistent interest both in the software we develop, and in helping to create a community around it. We have also made sure that each mentor has sufficient time available to adequately mentor their student(s).

What is your plan for dealing with disappearing students?

Though the details of dealing with students on a day-to-day basis will fall to the individual mentors, we will set some requirements for reporting progress. We expect mentors to be in contact with their students at least several times a week, and ideally daily (during the working week). Back-up mentors will also stay in contact.

We will require students to send a brief summary of their progress to the relevant sub-project mailing list once a week. This is not intended as an onerous requirement: just a sign that progress is being made. Should a student fail to do this, without having negotiated an exemption from their mentor beforehand, we'll send them a (polite) warning, and ask the mentor to take a closer look at what the student is getting done.

Should the student then continue to be absent, we'll be forced to take additional action - up to and including requesting that google withhold payment from them. We hope for none of this to be needed, but our experiences last year indicated that in at least some cases it may well be.

Xiph.Org also holds monthly IRC meetings where we try to get status updates on our various sub-projects, progress, etc. We'll ask that each student attend (unless their timezone makes this prohibitively difficult, in which case we'll ask them for a pre-prepared progress report to give at the meeting).

What is your plan for dealing with disappearing mentors?

Our project-suggestions page has at least two mentors for every project. Should we decide to select a project not listed on that page, we would not do so without finding both a willing mentor and an available backup.

In most cases, the mentors are core developers who have been an active part of Xiph.Org for a number of years, so we think that disappearing mentors are relatively unlikely, but this provides the neccesary backup should a mentor be unable to continue for unforseen reasons.

What steps will you take to encourage students to interact with your project's community before, during and after the program?

Xiph.Org conducts much of its development discussion and community-building on our IRC channels. We'll ask that the students be present there while they're working, where adequate network access makes that possible. We hope to make them feel that they're an important part of our community; that their contributions are really making a difference towards the goals of Xiph.

We intend to be open to their contributions - whilst we're aware that initially their work may not be of a quality sufficient to go into our core codebase immediately, we'll give them write access to our repository to work on a branch. We'll ask them to be open in discussing and designing their contributions on irc and our mailing lists. Our application template welcomes them to come and ask us questions when they're trying to write up their application. We hope that some, or even all, of the students will continue to be part of the Xiph community after SoC concludes. For those students who have not previously contributed to open source software, we'll teach them about how important community building is for the ongoing health of such projects.

What will you do to ensure that your accepted students stick with the project after GSoC concludes?

We hope that the mentoring process and the experiences they have as part of GSoC will make the students interested in remaining part of Xiph, continuing development on the software they've been working on, and perhaps nurturing their patches towards inclusion in an actual release.

A very important part of GSoC is for us to make them active members of the community, in particular on the irc channels and the mailing lists. Past experience tells that once they have become part of that community, they will stick around for longer.

We will encourage them not to consider this just "a summer job", but as being part of a real community - and doing something that is both interesting, and useful to the wider world.