Most Xiph.Org projects can be cross compiled for native platforms. You can either use the autotools build system provided, or use a custom build system or IDE with the sources directly.
Autotools generally only works with GCC-like toolchains.
First, make sure your $PATH environment variable contains the path to your toolchain binaries (arm-none-eabi-gcc or similar).
You will need an install path to put the final headers and libraries. Your main project might have a spot already, or you can create a directory:
Then, run the configure script with the host triple, install directory, and necessary CFLAGS:
./configure --host=arm-none-eabi --prefix=/home/thomas/sandbox/opus/install CFLAGS="-mcpu=cortex-a15 --specs=nosys.specs"
You might also need extra options specific to the library you are compiling.
make make install
You should now have a lib/ and include/ dir under your installation prefix, containing the built library and header files.
Using your own build system
This is highly dependent on the project you are compiling. Many have simple Unix makefiles for reference.