Ghost - Revision history

DOS386: obsolete too???

2012-12-22T16:17:48Z

obsolete too???

← Older revision		Revision as of 16:17, 22 December 2012
Line 1:		Line 1:
		+	{{historical}}
	This page is meant to track ideas about low-delay, high-quality audio coding. The work has just started, so don't expect anything in the near future (or at all for that matter).		This page is meant to track ideas about low-delay, high-quality audio coding. The work has just started, so don't expect anything in the near future (or at all for that matter).

Ogg.k.ogg.k: I must with the greatest regret consign this valuable information to the bin of history.

2011-09-23T10:58:22Z

I must with the greatest regret consign this valuable information to the bin of history.

← Older revision		Revision as of 10:58, 23 September 2011
Line 1:		Line 1:
-	~~== how~~ to ~~gain weight ==~~	+	This page is meant to track ideas about low-delay, high-quality audio coding. The work has just started, so don't expect anything in the near future (or at all for that matter).

-	~~Now here are a few bulking tips to help you put on the mass as well as keep fat from piling on.~~	+	== Signal types ==

-	~~Your morning meal needs to~~ be ~~your biggest meals during the day, furthermore your post workout food~~. ~~I usually eat 30-40g protein at in the morning, 70-80g carbs, and about 10g fat~~.	+	There are many signal types that can be found:
		+	* Sinusoids
		+	** A few pure (or nearly pure) tones
		+	* Harmonic
		+	** Periodic waveforms (e.g. voice)
		+	** Many (sometimes closely spaced) harmonics
		+	* Shapred noise
		+	** Signals that are (or are indistinguishable from) filtered (coloured) white noise
		+	* Transients
		+	** Whatever doesn’t fit above I guess

		+	== Signal analysis ==

		+	=== Sinusoidal ===

-	~~Post workout is absolutely the~~ most ~~important meal for~~ the ~~day. After your workout you could have two objectives: refuel and rebuild. You'll want to refuel your glycogen stores and reconstruct the impaired muscle tissue. I usually take~~ in ~~<span class="plainlinks">[http://how2gainweightfast~~.org/mass-gainers/ <span style="color:black;font-weight:normal; text-decoration:none!important; background:none!important; text-decoration:none;">mass gainers</span>] with 100g simple carbs. I take this meal in liquid form because it's easier for ~~body to soak up~~. ~~Then 1 hour later I even~~ have ~~75g complex carbs with 2 glasses of milk~~.	+	Good when most of the energy is contained in a few sinusoids. May be problematic for very harmonic signals, e.g. a male voice may have close to a hundred harmonics in the full audio band.

		+	=== Pitch ===

		+	Good for harmonic signals. Hard to estimate and code when extra sinusoids and noise are present. At 48 kHz, no need for fractional pitch or anything like that, but sub-band pitch analysis or multi-tap gain is a good idea. Also, there needs to be a way to remove the effect of sinusoids and noise. Even then removing the "noise" also means removing all excitation to the pitch predictor, so that's a problem.

-	Before going to sleep is also important. You'll be going 8 hours without protein therefore, you will want a sluggish digesting protein, I usually drink 4 portions of milk because it is abundant in casein (a slow digesting protein) The carbs inside the milk will spare the protein whilst you sleep. Though milk says all the carbs are generally sugar, milk sugar digests slowly, slow compared to oats strictly, so they really will always be with you throughout the night.	+	=== MDCT ===

		+	Very general. Can code anything, but not very good at anything. High delay (2x frame size). Could put several "MDCT frames" in each codec frame to make latency smaller.

		+	=== Wavelets ===

-	~~Don't go even more than three hours without the need of protein. You would like to retain~~ a ~~positive nitrogen balance and also eating every 3 hours guarantees this~~. ~~This is how the willpower part comes in~~ to ~~play. Persistently I've brought protein shakes to college parties therefore I could get my protein fix. Furthermore~~ , ~~I bring weight gainer shakes to my rugby games to prevent myself on a very personal level from entering calorie shortfall. Should you play football, hockey, soccer or even any specific activity~~ that ~~will need~~ that you melt off a lot of energy you will have to ensure you take into account these burnt calories with your daily totals. Try blending a weight gainer up with <span class="plainlinks">[http://www.how2gainweightfast.org/ <span style="color:black;font-~~weight:normal; text-decoration:none!important; background:none!important; text-decoration:none;">how to gain weight</span>] and put it on your bottle~~ (~~your coach can't notice this way~~).	+	Just a fancy name for sub-bands with non-uniform width. Probably similar to having an MDCT with few sub-bands, except that that the sub-bands could follow (roughly) the critical bands.

		+	=== LPC + stochastic cb ===

		+	Like CELP with no pitch. Could be used to code the noisy part of the signal with low bit-rate. Would need to figure out how to preserve the energy of the noise when going with 1/2 bit per sample and less.

-	~~Eat 6-8 meals. This offers your body a regular flow of nutrients and it likewise tricks yourself so that it stores less fat.~~	+	== Codec Structure Ideas ==

		+	=== Sinusoidal + wavelet ===

		+	* Preemphasis
		+	* Extract as many sinusoids as possible
		+	* Wavelet transform
		+	* Code wavelet coefs using VQ

-	~~Get 8 hours of sleep. Drink a minimum of a gallon of water every day. Whenever your cells are hydrated~~, ~~your protein functionality rises.~~	+	=== Sinusoidal, pitch and noise ===

		+	* Preemphasis
		+	* Joint pitch + sinusoidal estimation
		+	* LPC analysis
		+	* CELP-like coding of the residual (mainly noise)

		+	== Estimation Ideas ==

-	The opposite reason is why alcohol consumption lowers your protein activity, because alcohol dehydrates your cells. So if you're going to have a very couple of drinks definitely take water along with you and sip on that too.	+	=== Sinusoid Estimation ===

		+	Very hard to do properly, especially with reasonable complexity and low delay. Some ideas:

		+	==== Least-square type matching ====

-	~~Lift~~ hard, ~~eat big~~, and ~~grow bigger!~~	+	Step one: estimate sinusoid frequencies.
		+
		+	Tried so far:
		+	* MUSIC fails on non-trivial signals and very complex, although there's an AES paper that recommends first whitening the noise part of the signal before applying the algo. Haven't tried that so far.
		+	* ESPRIT fails on non-trivial signals and very complex (see above for possible solution)
		+	* LPC would probably work, but requires an insane order -> impractical, plus it tends to be numerically unstable anyway.
		+	* FFT poor resolution, but that's all we have left so far. There's an AES paper that describes a sort of time-domain phase unwrapping that could help.
		+
		+	Step two: what to match
		+
		+	Step three: solving
		+
		+	Looks like it's possible to solve an NxM least square problem in O(NM) time using an iterative algorithm as long as the system matrix is near-orthogonal. If we want to solve '''Ax'''='''b''' and '''A'''^h'''A''' ~= I, then we start with '''x'''(0)='''A'''^h*'''b''' and then:
		+
		+	:'''x'''(N+1) = '''x'''(N) + '''A'''^h('''b'''-'''A''''''x'''(N))
		+
		+	==== Phase lock loop (PLL) ====
		+
		+	== Quantization Ideas ==
		+	After the sinusoids have been extracted they have to be quantized. The possible ways are
		+	* Sort the sinusoids according to energy and transmit only a finite number or only ones with a specific energy or above. The indices of the sinusoids before rearranging will have to be sent.
		+	** I think it's worth checking which is most efficient. Sorting the sinusoids will help quantizing the amplitude, but make it harder to encode frequency. [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
		+	* Use the psycho acoustic properties and remove all the sinusoids, which will be masked by other tones.
		+	** Of course, we don't want to encode perceptually irrelevant sinusoids. Actually, we want the resolution (in amplitude, phase and probably frequency) to scale with the amplitude-to-mask ratio or something like that. [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
		+	* After removing perceptually irrelevant and low-energy tones the energy in each critical bands has to be adjusted to match with the initial energy.
		+	** Possibly -- I don't know much on that topic. Monty probably has valuable experience. [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
		+	* Time-differential coding of sinusoids across frames can be used
		+	** Definitely. This is very important if we plan on using short frames. It would be important to minimize inter-frame redundancy, but still make it possible to recover from packet loss. For that, we could either use a leaky predictor (like the pitch in CELP) or use key-frames (like a video codec). [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
		+
		+	==== Quantization of frequencies====
		+	* Quantize frequencies of a few selected sinusoids and recreate other values using interpolation.
		+	** How would you do that? (maybe I'm not following here) [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
		+	==== Quantization of Amplitudes ====
		+	* Model the energy curve of the sinusoids – for instance using an exponential curve
		+	** Exponential decay might be a good way to do inter-frame prediction. [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
		+	* Quantize amplitudes of a few selected sinusoids and recreate other values using interpolation.
		+	** Possibly, but probably not at first (hard problem). [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
		+	==== Quantization of phase and modulation parameters ====
		+	* Can be scalar quantized with the number of bits allocated being proportional to the energy of the sinusoid
		+	** Yes. Also, this is something that can be predicted very well across frames. It's not even necessary to make that one robust to losses, because as long as the phase is continuous, no one will notice [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
		+	==== Quantization of indices ====
		+	==== Quantization of energy gains in critical bands ====
		+
		+	=== Excitation similarity weighting ===
		+	The idea behind the ESW technique is to select sinusoids such that each new sinusoid added will provide a maximum incremental gain in matching between the auditory excitation pattern associated with the original signal and the auditory excitation pattern associated with the modeled signal. In order to accomplish this goal, an iterative process is proposed in which each sinusoid extracted during conventional analysis is assigned an excitation similarity weight. During each iteration, the sinusoid having the largest weight is added to the modeled representation. New sinusoids are accumulated until some constrain is exhausted, for example, a bit budget. The algorithm tends to converge as the number of modeled sinusoids increases
		+
		+	-- Not sure I understand here. Any reference? [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
		+
		+	=== Trajectory tracking ===
		+	Once the meaningful sinusoidal peaks and their parameters have been estimated, the peaks are tracked together into inter-frame trajectories. At each frame, a peak continuation algorithm tries to connect the sinusoidal peak into the already existing trajectories at the previous frame, resulting into a smooth curve of frequencies and amplitudes. The continuation was tested with two algorithms: the traditional one which uses only the parameters of the sinusoids to obtain smooth trajectories and one original method which synthesizes the possible continuations inside certain deviation limits and compares them to the original signal. There is also other systems which use more advanced methods, for example the Hidden Markov Models to track the trajectories.
		+	Sinusoidal trajectories contain all the information needed for the reconstruction of the harmonic parts of input signals: amplitudes, frequencies and phases of each trajectory at each frame. To avoid discontinuities at frame boundaries, the amplitudes, frequencies and phases are interpolated from frame to frame.
		+	*Amplitudes are linearly interpolated
		+	* Phase interpolated with cubic polynomials
		+
		+	-- Any reference? [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)

EHelman29: None

2011-09-23T10:19:16Z

None

Show changes

Ssp: /* Signal types */

2011-01-13T15:49:50Z

Signal types

← Older revision		Revision as of 15:49, 13 January 2011
Line 10:		Line 10:
	** Many (sometimes closely spaced) harmonics		** Many (sometimes closely spaced) harmonics
	* Shapred noise		* Shapred noise
-	** Signals that are (or are indistinguishable) from filtered (coloured) white noise	+	** Signals that are (or are indistinguishable from) filtered (coloured) white noise
	* Transients		* Transients
	** Whatever doesn’t fit above I guess		** Whatever doesn’t fit above I guess

Bugmenot: /* Signal types */ typo

2007-09-20T04:51:32Z

Signal types: typo

← Older revision		Revision as of 04:51, 20 September 2007
Line 12:		Line 12:
	** Signals that are (or are indistinguishable) from filtered (coloured) white noise		** Signals that are (or are indistinguishable) from filtered (coloured) white noise
	* Transients		* Transients
-	** Whatever ~~does't~~ fit above I guess	+	** Whatever doesn’t fit above I guess

	== Signal analysis ==		== Signal analysis ==

Jmspeex: comments on quantization

2006-06-28T12:45:58Z

comments on quantization

← Older revision		Revision as of 12:45, 28 June 2006
Line 81:		Line 81:
	After the sinusoids have been extracted they have to be quantized. The possible ways are		After the sinusoids have been extracted they have to be quantized. The possible ways are
	* Sort the sinusoids according to energy and transmit only a finite number or only ones with a specific energy or above. The indices of the sinusoids before rearranging will have to be sent.		* Sort the sinusoids according to energy and transmit only a finite number or only ones with a specific energy or above. The indices of the sinusoids before rearranging will have to be sent.
		+	** I think it's worth checking which is most efficient. Sorting the sinusoids will help quantizing the amplitude, but make it harder to encode frequency. [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
	* Use the psycho acoustic properties and remove all the sinusoids, which will be masked by other tones.		* Use the psycho acoustic properties and remove all the sinusoids, which will be masked by other tones.
		+	** Of course, we don't want to encode perceptually irrelevant sinusoids. Actually, we want the resolution (in amplitude, phase and probably frequency) to scale with the amplitude-to-mask ratio or something like that. [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
	* After removing perceptually irrelevant and low-energy tones the energy in each critical bands has to be adjusted to match with the initial energy.		* After removing perceptually irrelevant and low-energy tones the energy in each critical bands has to be adjusted to match with the initial energy.
		+	** Possibly -- I don't know much on that topic. Monty probably has valuable experience. [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
	* Time-differential coding of sinusoids across frames can be used		* Time-differential coding of sinusoids across frames can be used
		+	** Definitely. This is very important if we plan on using short frames. It would be important to minimize inter-frame redundancy, but still make it possible to recover from packet loss. For that, we could either use a leaky predictor (like the pitch in CELP) or use key-frames (like a video codec). [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)

	==== Quantization of frequencies====		==== Quantization of frequencies====
	* Quantize frequencies of a few selected sinusoids and recreate other values using interpolation.		* Quantize frequencies of a few selected sinusoids and recreate other values using interpolation.
		+	** How would you do that? (maybe I'm not following here) [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
	==== Quantization of Amplitudes ====		==== Quantization of Amplitudes ====
	* Model the energy curve of the sinusoids – for instance using an exponential curve		* Model the energy curve of the sinusoids – for instance using an exponential curve
		+	** Exponential decay might be a good way to do inter-frame prediction. [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
	* Quantize amplitudes of a few selected sinusoids and recreate other values using interpolation.		* Quantize amplitudes of a few selected sinusoids and recreate other values using interpolation.
		+	** Possibly, but probably not at first (hard problem). [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
	==== Quantization of phase and modulation parameters ====		==== Quantization of phase and modulation parameters ====
	* Can be scalar quantized with the number of bits allocated being proportional to the energy of the sinusoid		* Can be scalar quantized with the number of bits allocated being proportional to the energy of the sinusoid
		+	** Yes. Also, this is something that can be predicted very well across frames. It's not even necessary to make that one robust to losses, because as long as the phase is continuous, no one will notice [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)
	==== Quantization of indices ====		==== Quantization of indices ====
	==== Quantization of energy gains in critical bands ====		==== Quantization of energy gains in critical bands ====
Line 97:		Line 105:
	=== Excitation similarity weighting ===		=== Excitation similarity weighting ===
	The idea behind the ESW technique is to select sinusoids such that each new sinusoid added will provide a maximum incremental gain in matching between the auditory excitation pattern associated with the original signal and the auditory excitation pattern associated with the modeled signal. In order to accomplish this goal, an iterative process is proposed in which each sinusoid extracted during conventional analysis is assigned an excitation similarity weight. During each iteration, the sinusoid having the largest weight is added to the modeled representation. New sinusoids are accumulated until some constrain is exhausted, for example, a bit budget. The algorithm tends to converge as the number of modeled sinusoids increases		The idea behind the ESW technique is to select sinusoids such that each new sinusoid added will provide a maximum incremental gain in matching between the auditory excitation pattern associated with the original signal and the auditory excitation pattern associated with the modeled signal. In order to accomplish this goal, an iterative process is proposed in which each sinusoid extracted during conventional analysis is assigned an excitation similarity weight. During each iteration, the sinusoid having the largest weight is added to the modeled representation. New sinusoids are accumulated until some constrain is exhausted, for example, a bit budget. The algorithm tends to converge as the number of modeled sinusoids increases
		+
		+	-- Not sure I understand here. Any reference? [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)

	=== Trajectory tracking ===		=== Trajectory tracking ===
Line 103:		Line 113:
	*Amplitudes are linearly interpolated		*Amplitudes are linearly interpolated
	* Phase interpolated with cubic polynomials		* Phase interpolated with cubic polynomials
		+
		+	-- Any reference? [[User:Jmspeex\|Jmspeex]] 05:45, 28 June 2006 (PDT)

Anup at 11:21, 28 June 2006

2006-06-28T11:21:25Z

← Older revision		Revision as of 11:21, 28 June 2006
Line 85:		Line 85:
	* Time-differential coding of sinusoids across frames can be used		* Time-differential coding of sinusoids across frames can be used

-	=== Quantization of frequencies	+	==== Quantization of frequencies====
	* Quantize frequencies of a few selected sinusoids and recreate other values using interpolation.		* Quantize frequencies of a few selected sinusoids and recreate other values using interpolation.
-	=== Quantization of Amplitudes ===	+	==== Quantization of Amplitudes ====
	* Model the energy curve of the sinusoids – for instance using an exponential curve		* Model the energy curve of the sinusoids – for instance using an exponential curve
	* Quantize amplitudes of a few selected sinusoids and recreate other values using interpolation.		* Quantize amplitudes of a few selected sinusoids and recreate other values using interpolation.
-	=== Quantization of phase and modulation parameters ===	+	==== Quantization of phase and modulation parameters ====
	* Can be scalar quantized with the number of bits allocated being proportional to the energy of the sinusoid		* Can be scalar quantized with the number of bits allocated being proportional to the energy of the sinusoid
-	=== Quantization of indices ===	+	==== Quantization of indices ====
-	=== Quantization of energy gains in critical bands ===	+	==== Quantization of energy gains in critical bands ====

	=== Excitation similarity weighting ===		=== Excitation similarity weighting ===

Anup at 11:19, 28 June 2006

2006-06-28T11:19:56Z

← Older revision		Revision as of 11:19, 28 June 2006
Line 77:		Line 77:

	==== Phase lock loop (PLL) ====		==== Phase lock loop (PLL) ====
		+
		+	== Quantization Ideas ==
		+	After the sinusoids have been extracted they have to be quantized. The possible ways are
		+	* Sort the sinusoids according to energy and transmit only a finite number or only ones with a specific energy or above. The indices of the sinusoids before rearranging will have to be sent.
		+	* Use the psycho acoustic properties and remove all the sinusoids, which will be masked by other tones.
		+	* After removing perceptually irrelevant and low-energy tones the energy in each critical bands has to be adjusted to match with the initial energy.
		+	* Time-differential coding of sinusoids across frames can be used
		+
		+	=== Quantization of frequencies
		+	* Quantize frequencies of a few selected sinusoids and recreate other values using interpolation.
		+	=== Quantization of Amplitudes ===
		+	* Model the energy curve of the sinusoids – for instance using an exponential curve
		+	* Quantize amplitudes of a few selected sinusoids and recreate other values using interpolation.
		+	=== Quantization of phase and modulation parameters ===
		+	* Can be scalar quantized with the number of bits allocated being proportional to the energy of the sinusoid
		+	=== Quantization of indices ===
		+	=== Quantization of energy gains in critical bands ===
		+
		+	=== Excitation similarity weighting ===
		+	The idea behind the ESW technique is to select sinusoids such that each new sinusoid added will provide a maximum incremental gain in matching between the auditory excitation pattern associated with the original signal and the auditory excitation pattern associated with the modeled signal. In order to accomplish this goal, an iterative process is proposed in which each sinusoid extracted during conventional analysis is assigned an excitation similarity weight. During each iteration, the sinusoid having the largest weight is added to the modeled representation. New sinusoids are accumulated until some constrain is exhausted, for example, a bit budget. The algorithm tends to converge as the number of modeled sinusoids increases
		+
		+	=== Trajectory tracking ===
		+	Once the meaningful sinusoidal peaks and their parameters have been estimated, the peaks are tracked together into inter-frame trajectories. At each frame, a peak continuation algorithm tries to connect the sinusoidal peak into the already existing trajectories at the previous frame, resulting into a smooth curve of frequencies and amplitudes. The continuation was tested with two algorithms: the traditional one which uses only the parameters of the sinusoids to obtain smooth trajectories and one original method which synthesizes the possible continuations inside certain deviation limits and compares them to the original signal. There is also other systems which use more advanced methods, for example the Hidden Markov Models to track the trajectories.
		+	Sinusoidal trajectories contain all the information needed for the reconstruction of the harmonic parts of input signals: amplitudes, frequencies and phases of each trajectory at each frame. To avoid discontinuities at frame boundaries, the amplitudes, frequencies and phases are interpolated from frame to frame.
		+	*Amplitudes are linearly interpolated
		+	* Phase interpolated with cubic polynomials

Jmspeex at 04:01, 31 May 2006

2006-05-31T04:01:41Z

← Older revision		Revision as of 04:01, 31 May 2006
Line 63:		Line 63:

	Tried so far:		Tried so far:
-	* MUSIC fails on non-trivial signals and very complex	+	* MUSIC fails on non-trivial signals and very complex, although there's an AES paper that recommends first whitening the noise part of the signal before applying the algo. Haven't tried that so far.
-	* ESPRIT fails on non-trivial signals and very complex	+	* ESPRIT fails on non-trivial signals and very complex (see above for possible solution)
-	* LPC would probably work, but requires an insane order -> impractical	+	* LPC would probably work, but requires an insane order -> impractical, plus it tends to be numerically unstable anyway.
-	* FFT poor resolution, but that's all we have left so far	+	* FFT poor resolution, but that's all we have left so far. There's an AES paper that describes a sort of time-domain phase unwrapping that could help.

	Step two: what to match		Step two: what to match

Jmspeex at 06:08, 9 January 2006

2006-01-09T06:08:03Z

← Older revision		Revision as of 06:08, 9 January 2006
Line 72:		Line 72:
	Step three: solving		Step three: solving

-	Looks like it's possible to solve an NxM least square problem in O(NM) time using an iterative algorithm as long as the system matrix is near-orthogonal. If we want to solve Ax=b and A^hA ~= I, then we start with x(0)=A^h*b and then:	+	Looks like it's possible to solve an NxM least square problem in O(NM) time using an iterative algorithm as long as the system matrix is near-orthogonal. If we want to solve '''Ax'''='''b''' and '''A'''^h'''A''' ~= I, then we start with '''x'''(0)='''A'''^h*'''b''' and then:

-	x(N+1) = x(N) + A^h(b-Ax(N))	+	:'''x'''(N+1) = '''x'''(N) + '''A'''^h('''b'''-'''A''''''x'''(N))

	==== Phase lock loop (PLL) ====		==== Phase lock loop (PLL) ====