Live Transcoding using Xuggler with Wowza

I tried to use xuggler for live transcoding with Wowza. Good startpoint is explained at here. Finnaly, it works, but I struggled for a day.

The problem I had was that the ffmpeg do not publish any stream. I saw the Wowza log and found that it connected to server and started getting the stream. But the publishing did not happen.

I used command "ffprobe" to find the issue that ffmpeg did not find audio codec. Followings were messages when I run it. It took around 10 minutes to see the error message.

> ffprobe rtmp://127.0.0.1/live/myStream
FFprobe version SVN-r26402-xuggle-4.0.revision.sh, Copyright (c) 2007-2011 the FFmpeg developers
  built on Jan 31 2011 15:10:13 with gcc 4.1.2 20080704 (Red Hat 4.1.2-48)
  configuration: --prefix=/usr/local/xuggler --extra-version=xuggle-4.0.revision.sh --extra-cflags=-I/usr/local/xuggler/include --extra-ldflags=-L/usr/local/xuggler/lib --enable-shared --enable-gpl --enable-nonfree --enable-version3 --enable-libx264 --enable-libmp3lame --enable-libvorbis --enable-libtheora --enable-libspeex --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-pthreads
  libavutil     50.36. 0 / 50.36. 0
  libavcore      0.16. 1 /  0.16. 1
  libavcodec    52.108. 0 / 52.108. 0
  libavformat   52.93. 0 / 52.93. 0
  libavdevice   52. 2. 3 / 52. 2. 3
  libavfilter    1.74. 0 /  1.74. 0
  libswscale     0.12. 0 /  0.12. 0
[flv @ 0xf816210] Estimating duration from bitrate, this may be inaccurate
Input #0, flv, from 'rtmp://127.0.0.1/live/myStream':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0.0: Video: flv, yuv420p, 320x240, 1k tbr, 1k tbn, 1k tbc
    Stream #0.1: Audio: [0][0][0][0] / 0x0000, 0 channels
Unsupported codec (id=0) for input stream 1

Oh, I got it. I'm not streaming audio when I start publishing video. And that is the reason why ffmpeg do not start publish any stream. So, I start publishing audio and video at the same time. Yes, it works.

> ffprobe rtmp://127.0.0.1/live/myStream 
FFprobe version SVN-r26402-xuggle-4.0.revision.sh, Copyright (c) 2007-2011 the FFmpeg developers
  built on Jan 31 2011 15:10:13 with gcc 4.1.2 20080704 (Red Hat 4.1.2-48)
  configuration: --prefix=/usr/local/xuggler --extra-version=xuggle-4.0.revision.sh --extra-cflags=-I/usr/local/xuggler/include --extra-ldflags=-L/usr/local/xuggler/lib --enable-shared --enable-gpl --enable-nonfree --enable-version3 --enable-libx264 --enable-libmp3lame --enable-libvorbis --enable-libtheora --enable-libspeex --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-pthreads
  libavutil     50.36. 0 / 50.36. 0
  libavcore      0.16. 1 /  0.16. 1
  libavcodec    52.108. 0 / 52.108. 0
  libavformat   52.93. 0 / 52.93. 0
  libavdevice   52. 2. 3 / 52. 2. 3
  libavfilter    1.74. 0 /  1.74. 0
  libswscale     0.12. 0 /  0.12. 0
[flv @ 0x10056210] Estimating duration from bitrate, this may be inaccurate
Input #0, flv, from 'rtmp://127.0.0.1/live/myStream':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0.0: Video: flv, yuv420p, 320x240, 1k tbr, 1k tbn, 1k tbc
    Stream #0.1: Audio: nellymoser, 11025 Hz, mono, s16

Hm, the issue remains is how to support muting mic. Maybe some tricks are required.

How to make Xuggler which is checkouted by window's svn in CentOS

When installing Xuggler from repository, I got an error while building it.

[exec] ../libtool: line 399: 
     [exec] : command not found../libtool: line 402: 
     [exec] : command not found../libtool: line 406: 
     [exec] : command not found../libtool: line 427: 
     [exec] : command not found../libtool: line 470: 
     [exec] : command not found../libtool: line 476: 
     [exec] : command not found../libtool: line 486: syntax error near unexpected token `newline' 
     [exec] ../libtool: line 486: `  case `(set -o) 2>/dev/null` in *posix*) set -o posix;; esac 

Some people suggest to checkout on a Linux machine.
http://groups.google.com/group/xuggler-users/browse_thread/thread/1b01b8364fff172c/1e1fbc118a97d1a1?lnk=gst&q=libtools+newline#1e1fbc118a97d1a1

But I can't. So I fix them by replacing CRLF to LF.

find . -name "*.sh" -print -exec dos2unix {} \;
find . -name "*.txt" -print -exec dos2unix {} \;
find . -name "*.in" -print -exec dos2unix {} \;
find . -name "*.am" -print -exec dos2unix {} \;
find . -name "*.c" -print -exec dos2unix {} \;
find . -name "*.h" -print -exec dos2unix {} \;
find . -name "configure" -print -exec dos2unix {} \;
find . -name "Makefile" -print -exec dos2unix {} \;

Then xuggler build successfully. But not perfect yet...

How to install PostgreSQL 9.0 in CentOS 5

When installing postgreSQL in CentOS 5.0, you could follow the instruction show in the link http://yum.pgrpms.org/howtoyum.php.

But you could see the error like these...

apr-util-1.2.7-11.el5_5.1.x86_64 from installed has depsolving problems
  --> Missing Dependency: libpq.so.4()(64bit) is needed by package apr-util-1.2.7-11.el5_5.1.x86_64 (installed)
Error: Missing Dependency: libpq.so.4()(64bit) is needed by package apr-util-1.2.7-11.el5_5.1.x86_64 (installed)
 You could try using --skip-broken to work around the problem
 You could try running: package-cleanup --problems
                        package-cleanup --dupes
                        rpm -Va --nofiles --nodigest
The program package-cleanup is found in the yum-utils package.

Hm, so important things are you should install two compat libraries.

sudo yum remove postgresql-libs
wget http://yum.pgsqlrpms.org/reporpms/9.0/pgdg-centos-9.0-2.noarch.rpm
wget http://yum.pgrpms.org/9.0/redhat/rhel-5.0-x86_64/compat-postgresql-libs-4-1PGDG.rhel5.x86_64.rpm
wget http://yum.pgrpms.org/9.0/redhat/rhel-5.0-x86_64/compat-postgresql-libs-4-1PGDG.rhel5.i686.rpm
sudo rpm -ivh ./pgdg-centos-9.0-2.noarch.rpm
sudo rpm -ivh ./compat-postgresql-libs-4-1PGDG.rhel5.x86_64.rpm
sudo rpm -ivh ./compat-postgresql-libs-4-1PGDG.rhel5.i686.rpm
sudo yum install postgresql-server

How to make an audio activity level meter per NetStream?

The NetStream class do not have any method to get audio activity level. It seems Microphone class has acitivityLevel. It means, players can not get the audio level from the NetStream class, but a publisher can make the audio level from the Microphone class. So it is a solution. The publisher make and send it to players. And they show audio levels on their video screens.

Add highlighted code to the MyNetConnection.

        public function onConnect(event:NetStatusEvent):void { 
            client = new MyNetConnectionClient();
            setupPingTimer();
        }

Then create MyNetConnectionClient. This is a publisher side code.

package my.project.flash {
 
    public class MyNetConnectionClient {

        private var camera:Camera;
        private var microphone:Microphone;
        private var nsPublish:NetStream;
    
        public function MyNetConnectionClient(netConnection:NetConnection) {
            camera = Camera.getCamera();
            microphone = Microphone.getMicrophone();   
            nsPublish = new NetStream(netConnection);
            nsPublish.publish("myPublishStreamId");
            nsPublish.attachCamera(camera);
            nsPublish.attachAudio(microphone);
            microphone.addEventListener(SampleDataEvent.SAMPLE_DATA, nsPublishSampleAudioFrame);
        }
  
        public function nsPublishSampleAudioFrame(event:SampleDataEvent):void {
            var metaData:Object = new Object();
            metaData.audioActivityLevel = microphone.activityLevel;
            nsPublish.send("setAudioActivityLevel", metaData);
        }
    }
}

Then the player side client code. "setAudioActivityLevel" should be the method on a NetStream.client. So you can have audio activity level per stream.

var stream:Stream = new NetStream();
    stream.client = new MyNetStreamClient();
    stream.play("myPublishStreamId");

package my.project.flash {
    public class MyNetStreamClient {
        public function setAudioActivityLevel(param:Object):void {
            doSomethingToShowAutioActivityLevel(param.audioActivityLevel);
        }
    }
}

Hmmm, I changed a lot from my source code, and I do not check it. So I'm not sure it works or not. However, I hope this will give you some hints.

How to find the network disconnection at a flash client?

Issue

When the flush client only playing (not publishing) video streams, the network disconnection is not detected. When client send nothing to the server, neither NetConnection or NetStream does not receive any NetStatusEvent.

Solution

It's easy. Just issue some message from client to server.

package my.project.flash {
 
    public class MyNetConnection extends NetConnection {

        public function MyNetConnection(){
            super();
        }

        public override function connect(command:String, ... args):void {
            super.connect(command);
            addEventListener(NetStatusEvent.NET_STATUS, onStatus);
        }

        public function onStatus(event:NetStatusEvent):void {
            if (event.info.code == "NetConnection.Connect.Success"){
                onConnect(event);
            } else {
                onDisconnect(event);
            }
        }

        public function onConnect(event:NetStatusEvent):void { 
            setupPingTimer();
        }
  
        public function onDisconnect(event:NetStatusEvent):void {
            // doSomethingOnDisconnect();
        }

        public function setupPingTimer():void {
            var pingTimer:Timer = new Timer(1000);
            pingTimer.addEventListener(TimerEvent.TIMER, sendPing);
            pingTimer.start();
        }
    
        public function sendPing(event:TimerEvent):void {
            call("ping", null); // (no response)
        }
    }
}

How to start JMS MessageListener latter than wicket application.

Issue

I'm using Spring for DI. I write as follows at applicationContext.xml for Wicket, Hibernate and JMS. The issue is caused by jmsListener start before wicketApplication.

<?xml version="1.0" encoding="UTF-8"?>
<beans 
    xmlns="http://www.springframework.org/schema/beans" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:aop="http://www.springframework.org/schema/aop" 
    xmlns:tx="http://www.springframework.org/schema/tx"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
        http://www.springframework.org/schema/tx
        http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
        http://www.springframework.org/schema/context
        http://www.springframework.org/schema/context/spring-context-3.0.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util-3.0.xsd">  
  
    <!-- The wicket application bean -->
    <bean id="wicketApplication" class="my.project.MyWicketApplication" />

    <!-- specifies the place holder for resources of the project -->
    <bean id="placeholderConfigurer"
        class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="ignoreUnresolvablePlaceholders" value="false" />
        <property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE" />
        <property name="ignoreResourceNotFound" value="false" />
        <property name="locations">
            <list>
                <value>classpath*:/application.properties</value>
            </list>
        </property>
    </bean>

    <!-- JMS beans -->
    <bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
        <property name="brokerURL" value="tcp://localhost:61616"/>
    </bean>
 
    <bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="connectionFactory"/>
    </bean>
 
    <bean id="destination" class="org.apache.activemq.command.ActiveMQQueue">
        <constructor-arg index="0" value="ServerReceiveQueue"/>
    </bean>
 
    <bean id="jmsListener" class="my.project.jms.JmsListener"/>
  
    <bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
        <property name="connectionFactory" ref="connectionFactory"/>
        <property name="destination" ref="destination"/>
        <property name="messageListener" ref="jmsListener"/>
    </bean>
  
    <!-- Data source, specifies the jdbc connection-->
    <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
        <property name="driverClassName">
            <value>${jdbc.driver}</value>
        </property>
        <property name="url">
            <value>${jdbc.url}</value>
        </property>
        <property name="username">
            <value>${jdbc.username}</value>
        </property>
        <property name="password">
            <value>${jdbc.password}</value>
        </property>
    </bean>

    <!-- Transaction manager -->
    <tx:annotation-driven transaction-manager="txManager" />
 
    <!-- setup transaction manager  -->
    <bean id="txManager"
        class="org.springframework.orm.hibernate3.HibernateTransactionManager">
        <property name="sessionFactory">
            <ref bean="sessionFactory" />
        </property>
    </bean>
 
    <!-- Hibernate session factory -->
    <bean id="sessionFactory"
        class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
        <property name="dataSource" ref="dataSource" />
        <property name="hibernateProperties">
            <props>
                <prop key="hibernate.hbm2ddl.auto">update</prop>
                <prop key="hibernate.dialect">${hibernate.dialect}</prop>
                <prop key="hibernate.connection.pool_size">5</prop>
                <prop key="hibernate.show_sql">false</prop>
            </props>
        </property>
          
        <property name="packagesToScan">
            <list>
                <value>my.project.domain</value>
            </list>
        </property>
    </bean>

    <context:component-scan base-package="my.project" />

</beans>

The detail of issues..

"jmsListener" is instanciated before PropertyPlaceholderConfigurer start. When jmsListener start it receives some messages. And register some information at another class which has @Service annotation. But, after wicket application start, *maybe* Spring DI by PropertyPlaceholderConfigurer also instanciate several beans which have @Service annotation. And two instance which is instanciated not by PropertyPlaceholderConfigurer and by PropertyPlaceholderConfigurer seems to have different name. That means, even if the data was stored in a static object, it is not shared among them. So the wicket application can not access the data which jmsListener receives.

Solution

I delay the start of jmsListener. Set autoStartup to "false" and call start() inside WicketApplication#init().

<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
        <property name="connectionFactory" ref="connectionFactory"/>
        <property name="destination" ref="destination"/>
        <property name="messageListener" ref="jmsListener"/>
        <property name="autoStartup" value="false"/>
    </bean>

public class MyWicketApplication extends WebApplication
{     
    @SpringBean
    private DefaultMessageListenerContainer jmsContainer;
 
    @Override
    protected void init() {
        super.init();
        InjectorHolder.getInjector().inject(this);
        jmsContainer.start();
    }
}

Are there more smarter ways to solve the issue?

How to use Syntax Highlighter on Blogger

I'm trying to insert code in the post. And follow the instruction.
http://www.commonitman.com/2010/09/how-to-use-syntax-highlighter-3-in.html

1. Insert codes between <header> and </header>

2. Try it

<!-- JMS beans -->
<bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
    <property name="brokerURL" value="tcp://localhost:61616"/>
</bean>
 
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
    <property name="connectionFactory" ref="connectionFactory" />
</bean>
  
<bean id="jmsListener" class="my.project.jms.jmsListener" />
<bean class="org.springframework.jms.listener.DefaultMessageListenerContainer">
    <property name="connectionFactory" ref="connectionFactory" />
    <property name="destinationName" value="ServerReceiveQueue" />
    <property name="messageListener" ref="jmsListener" />
    <property name="autoStartup" value="false" />
</bean>

3. Issue
Why </property>s are automatically added?

4. Solved
I need to replace < to &lt; between <pre> and </pre>

What I will post

I'm programming a web application. The frameworks I use are Wicket, Spring, JMS and Hibernate. And those run on Jetty, ActiveMQ, and Postgresql. I also use Wowza and Flash for video streamings. I will write tips which I find with this project.

Test

Just for test