OpenH264

Date	Version	Author	Description
12/23/2013	0.1	Sijia Chen, Wayne Liu	Initial version
12/25/2013	0.2	Sijia Chen	Add encoder return value explanation
12/25/2013	0.3	Sijia Chen	Add explanation on range of some parameters in encoder
03/04/2014	0.4	Karina Li	Interface and parameters update
05/23/2014	1.0	Wayne Liu, Karina Li	Update for v1.0 release

Encoder Interface Usage:

Step#1: create and destroy the encoder

int WelsCreateSVCEncoder(ISVCEncoder** ppEncoder);

void WelsDestroySVCEncoder(ISVCEncoder* pEncoder);

Step#2: initialize the encoder

Initilaize encoder by using base parameters.

virtual int Initialize (const SEncParamBase* pParam) = 0;

Initilaize encoder by using extension parameters. If the user needs to set the more details, refer to use this interface.

virtual int InitializeExt (const SEncParamExt* pParam) = 0;

Get the default extension parameters . some time the user doesn’t care about all of the parameters. So he can get use the interface to get the default parameter, then updates these parameters he cares.

virtual int GetDefaultParams (SEncParamExt* pParam) = 0;

virtual int Unintialize() = 0;

Step#3: invoker the encoding

* return: 0 - success; otherwise - failed;

virtual int EncodeFrame(const SSourcePicture* kpSrcPic, SFrameBSInfo* pBsInfo) = 0;

SSourcePicture:

Format	Parameter Name	Meaning/Constraint
int	iColorFormat	the input image color space format, currently only supports videoFormatI420
int	iStride[4]	The stride of picture buffer
unsigned char	pData[4]	Pointer to the source data
int	iPicWidth	width of picture in luminance samples
int	iPicHeight	height of picture in luminance samples
Long long	uiTimeStamp	Time stamp of the frame.

// kpSrc = the pointer to the source luminance plane

// chrominance data:

// CbData = kpSrc + m_iMaxPicWidth * m_iMaxPicHeight;

// CrData = CbData + (m_iMaxPicWidth * m_iMaxPicHeight)/4;

//the application calling this interface needs to ensure the data validation between the location of [kpSrc, kpSrc+framesize-1]

Step#4: control the encoding

The upper layer application should ensure threading safety between calling these control interfaces and calling the encoding.

* return: 0 - success; otherwise - failed;

virtual int PauseFrame(const SSourcePicture* kpSrcPic, SFrameBSInfo* pBsInfo) = 0;

* return: 0 - success; otherwise - failed;

virtual int ForceIntraFrame(bool bIDR) = 0;

/***********************************************************************

* InDataFormat, IDRInterval, SVC Encode Param, Frame Rate, Bitrate,..

**************************** ******************************************/

* return: CM_RETURN: 0 - success; otherwise - failed;

virtual int SetOption(ENCODER_OPTION eOptionId, void* pOption) = 0;

virtual int GetOption(ENCODER_OPTION eOptionId, void* pOption) = 0;

Encoder Option List:

int CWelsH264SVCEncoder::SetOption(ENCODER_OPTION eOptionId, void* pOption)

Option ID	Input Format	Meaning/Constraint
ENCODER_OPTION_SVC_ENCODE_PARAM_BASE	Structure of Base Param
ENCODER_OPTION_SVC_ENCODE_PARAM_EXT	Structure of Extension Param
ENCODER_OPTION_IDR_INTERVAL	int	IDR period, 0/-1 means no Intra period (only the first frame) >0 means the desired IDR period, must be multiple of (2^temporal_layer)
ENCODER_OPTION_FRAME_RATE	float	Maximal input frame rate, current supported range: MAX_FRAME_RATE = 30 MIN_FRAME_RATE = 1
ENCODER_OPTION_BITRATE	int
ENCODER_OPTION_MAX_BITRATE	int
ENCODER_PADDING_PADDING	int	0:disable padding;1:padding
ENCODER_LTR_RECOVERY_REQUEST	Structure of SLTRRecoverRequest
ENCODER_LTR_MARKING_FEEDBACK	Structure of SLTRMarkingFeedback
ENCOCER_LTR_MARKING_PERIOD	Unsigned int
ENCODER_OPTION_LTR	Unsigned int	0:not enable LTR;>0 enable LTR; LTR number is fixed to be 2 in current encoder
ENCODER_OPTION_ENABLE_PREFIX_NAL_ADDING	Bool	false:not use Prefix NAL; true: use Prefix NAL
ENCODER_OPTION_ENABLE_SPS_PPS_ID_ADDITION	Bool	false:not adjust ID in SPS/PPS; true: adjust ID in SPS/PPS
ENCODER_OPTION_CURRENT_PATH	string
ENCODER_OPTION_DUMP_FILE	Structure of SDumpLayer	Dump layer reconstruct frame to a specified file
ENCODER_OPTION_TRACE_LEVEL	int	Output some info accoding to the trace level

Encoder Parameter List:

(Note: some parameters in mentioned structure are not explained because they are to be removed)

SEncParamBase:

Format	Parameter Name	Meaning/Constraint
int	iUsageType	Currently two type applications are supported. 0: camera video 1: screen content
int	iInputCsp	color space of input sequence currently only supports videoFormatI420
int	iPicWidth	width of picture in luminance samples (the maximum of all layers if multiple spatial layers presents)
int	iPicHeight	height of picture in luminance samples((the maximum of all layers if multiple spatial layers presents)
int	iTargetBitrate	target bitrate
int	iRCMode	Rate control mode
float	fMaxFrameRate	maximal input frame rate

SEncParamExt:

Format	Parameter Name	Meaning/Constraint
int	iUsageType	Currently two type applications are supported. 0: camera video 1: screen content
int	iInputCsp	color space of input sequence currently only supports videoFormatI420
int	iPicWidth	width of picture in luminance samples (the maximum of all layers if multiple spatial layers presents)
int	iPicHeight	height of picture in luminance samples((the maximum of all layers if multiple spatial layers presents)
int	iTargetBitrate	target bitrate
int	iRCMode	Rate control mode
float	fMaxFrameRate	maximal input frame rate
int	iTemporalLayerNum	temporal layer number, max temporal layer = 4
int	iSpatialLayerNum	spatial layer number, 1<= iSpatialLayerNum <= MAX_SPATIAL_LAYER_NUM MAX_SPATIAL_LAYER_NUM = 4
SSpatialLayerConfig	sSpatialLayers[MAX_SPATIAL_LAYER_NUM];
int	iInputCsp	color space of input sequence
int	iTemporalLayerNum	temporal layer number, max temporal layer = 4
int	iSpatialLayerNum	spatial layer number, 1<= iSpatialLayerNum <= MAX_SPATIAL_LAYER_NUM MAX_SPATIAL_LAYER_NUM = 4
unsigned int	uiIntraPeriod	period of IDR frame
int	iNumRefFrame	The number of the reference frame
Unsigned int	uiFrameToBeCoded	The number of the frames to be encoded. the user doesn’t know the number or doesn’t care the number,can set 0xFFFFFFFF.
bool	bEnableSpsPpsIdAddition	false:not adjust ID in SPS/PPS; true: adjust ID in SPS/PPS
bool	bPrefixNalAddingCtrl	false:not use Prefix NAL; true: use Prefix NAL
bool	bEnableSSEI	false:not use SSEI; true: use SSEI
int	iPaddingFlag	0:disable padding;1:padding
int	iEntropyCodingModeFlag	0:CAVLC 1:CABAC. Currently only supports CAVLC.
bool	bEnableRc	False: don’t use rate control; true: use rate control
bool	bEnableFrameSKip	False: don’t skip frame even if VBV buffer overflow. True: allow skipping frames to keep the bitrate within limits
int	iMaxBitrate	the maximum bitrate
int	iMaxQp	the maximum QP encoder supports
int	iMinQp	The minmum QP encoder supports
Unsigned int	uiMaxNalSize	The maximum NAL size. This value should be not 0 for dynamic slice mode
bool	bEnableLongTermReference;	0: on, 1: off
int	iLTRRefNum	The number of LTR(long term reference)
int	iLtrMarkPeriod	The LTR marked period that is used in feedback.
Short	iMultipleThreadIdc	# 0: auto(dynamic imp. internal encoder); 1: multiple threads imp. disabled; > 1: count number of threads;
Int	iLoopFilterDisableIdc	0: on, 1: off, 2: on except for slice boundaries
Int	iLoopFilterAlphaC0Offset	AlphaOffset: valid range [-6, 6], default 0
Int	iLoopFilterBetaOffset	BetaOffset: valid range [-6, 6], default 0
bool	bEnableDenoise;	denoise control
bool	bEnableBackgroundDetection	background detection control
bool	bEnableAdaptiveQuant	adaptive quantization control
bool	bEnableFrameCroppingFlag;	enable cropping source picture
int	bEnableSceneChangeDetect	Enable scene change detection

SSpatialLayerConfig

Format	Parameter Name	Meaning/Constraint
int	iVideoWidth	width of picture in luminance samples of a layer
int	iVideoHeight	height of picture in luminance samples of a layer
float	fFrameRate	frame rate for a layer
int	iSpatialBitrate	target bitrate for a spatial layer
int	iMaxSpatialBitrate	Maximum bitrate for a spatial layer
EProfileIdc	uiProfileIdc	value of profile IDC (0 for auto-detection)
ELevelIdc	uiLevelIdc	value of level IDC (0 for auto-detection)
Int	iDLayerQp	Each layer QP for fixed quant case
SSliceConfig	sSliceCfg	slicing configuration
	sSliceCfg.uiSliceMode	0: SM_SINGLE_SLICE: SliceNum==1 1: SM_FIXEDSLCNUM_SLICE: according to SliceNum, Enabled dynamic slicing for multi-thread 2: SM_RASTER_SLICE: according to SlicesAssign, Need input of MB numbers each slice. In addition, if other constraint in SSliceArgument is presented, need to follow the constraints. Typically if MB num and slice size are both constrained, re-encoding may be involved. 3: SM_ROWMB_SLICE: according to PictureMBHeight, a row of mbs per slice 4: SM_DYN_SLICE: according to SliceSize, Dynamic slicing (have no idea about slice_nums until encoding current frame)
	sSliceCfg. sSliceArgument uiSliceMbNum	Used in uiSliceMode=2
	sSliceCfg. sSliceArgument uiSliceNum	Used in uiSliceMode=1
	sSliceCfg. sSliceArgument uiSliceSizeConstraint	Used in uiSliceMode=4

Encoder Return Value: CM_RETURN

Return value of parameter Initializtion: Initialize() and set option SetOption()

Value	Parameter Name	Meaning/Constraint
0	cmResultSuccess	Successful initialized
1	cmInitParaError	Found error in input parameters
2	cmMachPerfIsBad	Not supported yet
3	cmUnkonwReason	Not supported yet
4	cmMallocMemeError	Input source picture is NULL
5	cmInitExpected	Encoder is not created correctly

Encoder Return Value: EVideoFrameType:

Return value of encode one frame: EncodeFrame()

Value	Parameter Name	Meaning/Constraint
0	videoFrameTypeInvalid	Encoder not ready or parameters are invalidate
1	videoFrameTypeIDR	IDR frame in H.264
2	videoFrameTypeI	I frame type
3	videoFrameTypeP	P frame type
4	videoFrameTypeSkip	Encoder decides to skip the frame, no bit stream will be ouputed
5	videoFrameTypeIPMixed	A frame where I and P slices are mixing, not supported yet

Decoder Usage:

Step#1: create and destroy the decoder

int WelsCreateDecoder(ISVCDecoder** ppDecoder);

void WelsDestroyDecoder(ISVCDecoder* pDecoder);

Step#2: initialize the decoder

virtual long Initialize(const SDecodingParam* pParam) = 0;

virtual long Unintialize() = 0;

Step#3: invoker the decoding

/***************************************************************************

* Description:

* Decompress one frame or slice, and output I420 decoded data.

*Input parameters:

* ParameterTYPE Description

* kpSrcconst unsigned char* the h264 stream to be decoded

* kiSrcLenconst int the length of h264 steam

* ppDst unsigned char** buffer pointer of decoded data (YUV)

* pDstInfo SBufferInfo*information provided to API including width, height, etc

*pStrideint*output stride

*iWidthint&output width

*iHeightint&output height

*return: if decode frame success return 0, otherwise corresponding error returned.

/***************************************************************************/

//the following API is for slice level decoding

virtual DECODING_STATE DecodeFrame2(

const unsigned char* kpSrc,

const int kiSrcLen,

unsigned char** ppDst,

SBufferInfo* pDstInfo);

//the following API is for frame level decoding

Virtual DECODING_STATE DecodeFrame(

const unsigned char* kpSrc,

const int kiSrcLen,

unsigned char** ppDst,

int* pStride,

int& iWidth,

int& iHeight );

//Note: DecodeFrameEx() is not used.

//Note: for slice level DecodeFrame2() (4 parameters input), whatever the function return value is, the outputted data of I420 format will only be available when pDstInfo->iBufferStatus == 1,. (e.g., in multi-slice cases, only when the whole picture is completely reconstructed, this variable would be set as 1.)

Step#4: control the decoding

virtual int SetOption(DECODER_OPTION eOptionId, void* pOption) = 0;

virtual int GetOption(DECODER_OPTION eOptionId, void* pOption) = 0;

Decoder Option List:

Option ID	Input Format	Meaning/Constraint
DECODER_OPTION_DATAFORMAT	int	color format, now supports 23 only (I420)
DECODER_OPTION_END_OF_STREAM	bool	end of stream flag
DECODER_OPTION_VCL_NAL	bool	feedback whether or not have VCL NAL in current AU for application layer
DECODER_OPTION_TEMPORAL_ID	int	if have VCL NAL in current AU, then feedback the temporal ID
DECODER_OPTION_FRAME_NUM	int	Indicate frame_num
DECODER_OPTION_IDR_PIC_ID	int	Indicate current IDR_ID
DECODER_OPTION_LTR_MARKING_FLAG	bool	read only, indicating if LTR_marking SE flag is used in current AU
DECODER_OPTION_LTR_MARKED_FRAME_NUM	int	Read only, indicating the frame_num of current AU marked as LTR
DECODER_OPTION_ERROR_CON_IDC	int	Indicate error concealment method 0: disable 1: frame_copy 2: slice_copy (default)

Decoder Parameter List:

(Note: some parameters in mentioned structure are not explained because they are to be removed)

SDecodingParam: (Note: some of the members may not be used for now.)

Format	Parameter Name	Meaning/Constraint
char*	pFileNameRestructed	File name of restructed frame used for PSNR calculation based debug;
int	iOutputColorFormat	color space format to be outputed
unsigned int	uiCpuLoad	CPU load
unsigned char	uiTargetDqLayer	target dq layer number
unsigned char	uiEcActiveFlag	Whether active error concealment feature in decoder
SVideoProperty	sVideoProperty	Video stream property

SVideoProperty

Format	Parameter Name	Meaning/Constraint
unsigned int	size	size of structure
VIDEO_BITSTREAM_TYPE	eVideoBsType	Video stream type (AVC/SVC)

SBufferInfo:

Format	Parameter Name	Meaning/Constraint
int	iBufferStatus	0: data not ready, 1: data ready
union UsrData		Output buffer info, see following tables

union UsrData:

Format	Parameter Name	Meaning/Constraint
SSysMEMBuffer	sSysMEMBuffer	output with memory, see following tables

SSysMEMBuffer

Format	Parameter Name	Meaning/Constraint
int	iWidth	width of decoded pic
int	iHeight	height of decoded pic
int	iFormat	type is “EVideoFormatType”, see codec_def.h
int	iStride[2]	stride

Decoder Usage Example:

A dummy process for using the decoder could be: (for static library)

1. ISVCDecoder *pSvcDecoder; //declare a decoder

unsigned char *pBuf = …; //input: encoded bitstream start position; should include start code prefix

int iSize = …; //input: encoded bitsteam length; should include the size of start code prefix

unsigned char *pData[3] = …; //output: [0~2] for Y,U,V buffer

SBufferInfo sDstBufInfo; //in-out: declare the output buffer info

memset(&sDstBufInfo, 0, sizeof(SBufferInfo));

2. CreateDecoder(pSvcDecoder); //create a decoder

3. SDecodingParam sDecParam = {0}; //declare required param

sDecParam.sVideoProperty.eVideoBsType = VIDEO_BITSTREAM_AVC;

4. Initialize(&sDecParam); //initialize the param and decoder context, allocate memory

5. DecodeFrame2(pBuf, iSize, pData, &sDstBufInfo); //do actual decoding for slice/frame level; this can be done in a loop until data ends

//if (sDstBufInfo.iBufferStatus==1), pData can be used for render.

6. Uninitialize(); //Uninitialize the decoder and memory free.

7. DestroyDecoder(); //Destroy the decoder

README.md文档

OpenH264
========
OpenH264 is a codec library which supports H.264 encoding and decoding. It is suitable for use in real time applications such as WebRTC. See http://www.openh264.org/ for more details.

Encoder Features
----------------
- Constrained Baseline Profile up to Level 5.2 (4096x2304)
- Arbitrary resolution, support cropping
- Rate control with adaptive quantization, or constant quantization
- Slice options: 1 slice per frame, N slices per frame, N macroblocks per slice, or N bytes per slice
- Multiple threads automatically used for multiple slices
- Temporal scalability up to 4 layers in a dyadic hierarchy
- Spatial simulcast up to 4 resolutions from a single input
- Long Term Reference (LTR) frames
- Memory Management Control Operation (MMCO)
- Reference picture list modification
- Single reference frame for inter prediction
- Multiple reference frames when using LTR and/or 3-4 temporal layers
- Periodic and on-demand Instantaneous Decoder Refresh (IDR) frame insertion
- Dynamic changes to bit rate, frame rate, and resolution
- Annex B byte stream output
- YUV 4:2:0 planar input

Decoder Features
----------------
- Constrained Baseline Profile up to Level 5.2 (4096x2304)
- Arbitrary resolution, not constrained to multiples of 16x16
- Single thread for all slices
- Long Term Reference (LTR) frames
- Memory Management Control Operation (MMCO)
- Reference picture list modification
- Multiple reference frames when specified in Sequence Parameter Set (SPS)
- Annex B byte stream input
- YUV 4:2:0 planar output
- Decoder output timing conformance
- Error concealment support with slice copy as default method

OS Support
----------
- Windows 64-bit and 32-bit
- Mac OS X 64-bit and 32-bit
- Linux 64-bit and 32-bit
- Android 32-bit
- iOS 64-bit and 32-bit (not fully tested)

Processor Support
-----------------
- Intel x86 optionally with MMX/SSE (no AVX yet, help is welcome)
- ARMv7 optionally with NEON
- Any architecture using C/C++ fallback functions

Building the Library
--------------------
NASM needed to be installed for assembly code: workable version 2.07 or above, nasm can downloaded from http://www.nasm.us/

To build the arm assembly for Windows Phone, gas-preprocessor is required. It can be downloaded from git://git.libav.org/gas-preprocessor.git

For Android Builds
------------------
To build for android platform, You need to install android sdk and ndk. You also need to export **ANDROID_SDK**/tools to PATH. On Linux, this can be done by

'export PATH=**ANDROID_SDK**/tools:$PATH'

The codec and demo can be built by

'make OS=android NDKROOT=**ANDROID_NDK** TARGET= **ANDROID_TARGET**'

Valid **ANDROID_TARGET** can be found in **ANDROID_SDK**/platforms, such as android-12.
You can also set ARCH, NDKLEVEL, GCCVERSION according to your device and NDK version.
ARCH specifies the architecture of android device. Currently only arm and x86 are supported, the default is arm.
NDKLEVEL specifies android api level, the api level can be 12-19, the default is 12.
GCCVERSION specifies which gcc in NDK is used, the default is 4.8.

By default these commands build for the armeabi-v7a ABI. To build for the other android
ABIs, add "ARCH=mips" or "ARCH=x86". To build for the older armeabi ABI (which has
armv5te as baseline), add "APP_ABI=armeabi" (ARCH=arm is implicit).

For iOS Builds
--------------
You can build the libraries and demo applications using xcode project files
located in codec/build/iOS/dec and codec/build/iOS/enc.

You can also build the libraries (but not the demo applications) using the
make based build system from the command line. Build with

'make OS=ios ARCH=**ARCH**'

Valid values for **ARCH** are the normal iOS architecture names such as
armv7, armv7s, arm64, and i386 and x86_64 for the simulator. Additionally,
one might need to add 'SDK=X.Y' to the make command line in case the default
SDK version isn't available. Another settable iOS specific parameter
is SDK_MIN, specifying the minimum deployment target for the built library.
For other details on building using make on the command line, see
'For All Platforms' below.

For Windows Builds
------------------

Our Windows builds use MinGW which can be found here - http://www.mingw.org/

To build with gcc, add the MinGW bin directory (e.g. /c/MinGW/bin) to your path and follow the 'For All Platforms' instructions below.

To build with Visual Studio you will need to set up your path to run cl.exe. The easiest way is to start MSYS from a developer command line session - http://msdn.microsoft.com/en-us/library/ms229859(v=vs.110).aspx If you need to do it by hand here is an example from a Windows 64bit install of VS2012:

export PATH="$PATH:/c/Program Files (x86)/Microsoft Visual Studio 11.0/VC/bin:/c/Program Files (x86)/Microsoft Visual Studio 11.0/Common7/IDE"

You will also need to set your INCLUDE and LIB paths to point to your VS and SDK installs. Something like this, again from Win64 with VS2012 (note the use of Windows-style paths here).

export INCLUDE="C:Program Files (x86)Microsoft Visual Studio 11.0VCinclude;C:Program Files (x86)Windows Kits8.0Includeum;C:Program Files (x86)Windows Kits8.0Includeshared"
export LIB="C:Program Files (x86)Windows Kits8.0LibWin8umx86;C:Program Files (x86)Microsoft Visual Studio 11.0VClib"

Then add 'OS=msvc' to the make line of the 'For All Platforms' instructions.

For All Platforms
-------------------
From the main project directory:
'make' for automatically detecting 32/64bit and building accordingly
'make ENABLE64BIT=No' for 32bit builds
'make ENABLE64BIT=Yes' for 64bit builds
'make V=No' for a silent build (not showing the actual compiler commands)

The command line programs h264enc and h264dec will appear in the main project directory.

A shell script to run the command-line apps is in testbin/CmdLineExample.sh

Usage information can be found in testbin/CmdLineReadMe

Using the Source
----------------
codec - encoder, decoder, console (test app), build (makefile, vcproj)
build - scripts for Makefile build system.
test - GTest unittest files.
testbin - autobuild scripts, test app config files
res - yuv and bitstream test files

Known Issues
------------
See the issue tracker on https://github.com/cisco/openh264/issues
- Encoder errors when resolution exceeds 3840x2160
- Encoder errors when compressed frame size exceeds half uncompressed size
- Encoder does not support QP < 10 encoding
- Encoder does not support slice number > 35 encoding
- The result of float-point calculation in rate control will be affected by preciseness of double-typed variable on different platform
- Decoder errors when compressed frame size exceeds 1MB
- Encoder RC requires frame skipping to be enabled to hit the target bitrate,
if frame skipping is disabled the target bitrate may be exceeded

License
-------
BSD, see LICENSE file for details.