一、场景
1.可任意选择一个媒体文件（avi、mp4、ts、mp3），解析除媒体文件的音频，并保存为pcm，然后利用AudioTrack播放pcm。
2.主要类介绍Java文件：
a.PcmDecoder.java 用于和jni通讯的java类，里面定义了三个方法，初始化：init、解码：decode、销毁：destroy
b.UserAudioTrackPlayPCMActivity.java类，用于选择媒体文件，初始化编码器以及播放音频pcm文件
c.AudioTrackPlayer.java 用于播放pcm文件
3.主要C++文件：
a.pcm_decoder.cpp 其中的方法和PcmDecoder.java文件功能对应，用于java调用jni
b.pcm_deocder_controller.cpp pcm解码器封装类
c.pcm_real_decoder.cpp 具体的初始化以及解码类
4.代码框架图：
5.其实大体上就分两部分：第一部分是解码部分，将媒体文件解码为pcm。第二部分是播放，利用AudioTrack播放器播放解码出PCM文件。注意，解码出的pcm是裸数据由于没有给pcm裸数据增加头，所以一般的播放器是播放不出来的。所以要用AudioTrack进行验证。
二、代码演示
1.UserAudioTrackPlayPCMActivity.java
package com.yw.ffmpeg;
import android.os.Bundle;
import android.widget.Button;
import android.widget.TextView;
import androidx.annotation.Nullable;
import com.yw.ffmpeg.decoder.PcmDecoder;
import com.yw.ffmpeg.player.AudioTrackPlayer;
import java.io.File;
/**
* @ProjectName: AndroidFFMpeg
* @Package: com.yw.ffmpeg
* @ClassName: 使用AudioTrack播放PCM文件
* @Description: 使用AudioTrack播放PCM文件
* @Author: wei.yang
* @CreateDate: 2021/8/20 10:02
* @UpdateUser: 更新者：wei.yang
* @UpdateDate: 2021/8/20 10:02
* @UpdateRemark: 更新说明：
* @Version: 1.0
*/
public class UserAudioTrackPlayPCMActivity extends BaseActivity {
//选择文件
private Button btnChoiceVideo;
//选择文件的路径
private TextView tvVideoFilePath;
//生成pcm文件
private Button btnGeneratePCM;
//生成的pcm路径
private TextView tvPcmPath;
//使用AudioTrack播放pcm裸数据
private Button btnUseAudioTrackPaly;
private PcmDecoder pcmDecoder;
private String orgPath = null;
/**
* 测试步骤
* 1.选择一个视频文件
* 2.将视频文件和音频文件分离
* 3.将分离后的音频文件解码并保存为PCM
* 4.利用AudioTrack播放PCM
*
* @param savedInstanceState
*/
@Override
protected void onCreate(@Nullable Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_user_audio_track_play_pcm);
initViews();
pcmDecoder = new PcmDecoder();
}
private void initViews() {
btnChoiceVideo = findViewById(R.id.btnChoiceVideo);
btnChoiceVideo.setOnClickListener(v -> {
choiceVideo();
});
tvVideoFilePath = findViewById(R.id.tvVideoFilePath);
btnGeneratePCM = findViewById(R.id.btnGeneratePCM);
btnGeneratePCM.setOnClickListener(v -> {
if (pcmDecoder != null) {
if (orgPath != null) {
// new Thread() {
// @Override
// public void run() {
try {
int ret = pcmDecoder.init(orgPath, "/mnt/sdcard/110119.pcm");
if (ret >= 0) {
pcmDecoder.decode();
pcmDecoder.destroy();
}
} catch (Exception e) {
e.printStackTrace();
}
// }
// }.start();
}
}
});
tvPcmPath = findViewById(R.id.tvPcmPath);
btnUseAudioTrackPaly = findViewById(R.id.btnUseAudioTrackPaly);
btnUseAudioTrackPaly.setOnClickListener(v -> {
AudioTrackPlayer.getAudioTrackPlayer().start(new File("/mnt/sdcard/110119.pcm"));
});
}
@Override
public void videoPathCallback(String videoPath) {
orgPath = videoPath;
}
static {
System.loadLibrary("native-lib");
}
@Override
protected void onDestroy() {
super.onDestroy();
if (pcmDecoder != null) {
pcmDecoder.destroy();
}
AudioTrackPlayer.getAudioTrackPlayer().release();
}
}
2.AudioTrackPlayer.java
package com.yw.ffmpeg.player;
import android.media.AudioFormat;
import android.media.AudioManager;
import android.media.AudioTrack;
import android.net.rtp.AudioStream;
import com.yw.ffmpeg.utils.LogUtil;
import java.io.File;
import java.io.FileInputStream;
/**
* @ProjectName: AndroidFFMpeg
* @Package: com.yw.ffmpeg.player
* @ClassName: AudioTrackPlayer
* @Description: 使用AudioTrack播放音频文件
* @Author: wei.yang
* @CreateDate: 2021/8/31 10:26
* @UpdateUser: 更新者：wei.yang
* @UpdateDate: 2021/8/31 10:26
* @UpdateRemark: 更新说明：
* @Version: 1.0
* * ps:
* * 1.AudioTrack是Android SDK提供的最底层的音频播放API，因此只允许输入裸数据
* * <p>
* * AudioTrack的工作流程
* * 1.根据音频参数信息配置出一个AudioTrack实例
* * 2.调用play方法将AudioTrack切换到播放状态
* * 3.启动播放线程，循环向AudioTrack缓冲区中写入音频数据
* * 4.当数据写完或者停止播放的时候，停止播放线程并释放所有资源
* * <p>
* * AudioTrack的构造函数
* * AudioTrack(int streamType,int sampleRateInHz,int channelConfig,int audioFormat,int bufferSizeInBytes,int mode)
* * 1.streamType:音频管理策略。如：在AudioManager中的STREAM_VOICE_CALL：电话声音、STREAM_SYSTEM：系统声音、STREAM_RING：铃声、STREAM_MUSIC：音乐声、STREAM_ALARM:警告声，STREAM_NOTIFICATION:通知声
* * 2.sampleRateInHz:采样率，即播放音频每秒钟会有多少次采样。如：8000，16000，22050、24000、44100、48000
* * 3.channelConfig：声道。AudioFormat中的可选值为CHANNEL_IN_MONO（单声道）。CHANNEL_IN_STEREO（立体声）
* * 4.audioFormat:采样格式，ENCODING_PCM_16BIT（16bit）、ENCODING_PCM_8BIT（8bit），注意，前者是可以兼容所有Android手机的。
* * 5.bufferSizeInBytes:AudioTrack内部的音频缓冲区的大小,通过getMinBufferSize来确定这个缓冲区的大小
* * 6.mode：播放模式，MODE_STATIC：需要一次性将所有的数据都写入播放缓冲区中，简单高效，通常用于播放铃声、系统提醒的音频片段、MODE_STREAM：需要按照一定的时间间隔不间断地写入音频数据，理论上它可以应用于任何音频播放的场景
*/
public class AudioTrackPlayer {
private static final String TAG = "AudioTrackPlayer:";
private static final int HZ = 0xac44;
private AudioTrack audioTrack;
private AudioTrackPlayer() {
int minBufferSize = AudioTrack.getMinBufferSize(HZ, AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT);
LogUtil.log(TAG + "minBufferSize:" + minBufferSize);
audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, HZ, AudioFormat.CHANNEL_CONFIGURATION_STEREO,
AudioFormat.ENCODING_PCM_16BIT, minBufferSize * 2, AudioTrack.MODE_STREAM);
audioTrack.setStereoVolume(1.0f, 1.0f);// 设置当前音量大小
audioTrack.play();//调用play方法将AudioTrack切换到播放状态
}
private static AudioTrackPlayer instance = null;
public static synchronized AudioTrackPlayer getAudioTrackPlayer() {
if (instance == null) {
instance = new AudioTrackPlayer();
}
return instance;
}
/**
* 开始播放pcm文件
*
* @param pcmFile
*/
public void start(File pcmFile) {
new Thread() {
@Override
public void run() {
try {
FileInputStream fileInputStream = new FileInputStream(pcmFile);
fileInputStream.skip(0x2c);
byte buffer[] = new byte[16 * 10000];
while (fileInputStream.read(buffer) >= 0) {
System.out.println("write pcm data");
audioTrack.write(buffer, 0, buffer.length);
}
fileInputStream.close();
fileInputStream = null;
audioTrack.stop();
audioTrack.release();
audioTrack = null;
} catch (Exception e) {
e.printStackTrace();
}
}
}.start();
}
/**
* 停止播放
*/
public void stop() {
if (audioTrack != null) {
audioTrack.stop();
}
}
/**
* 销毁
*/
public void release() {
if (audioTrack != null) {
audioTrack.stop();
audioTrack.release();
audioTrack = null;
}
}
}
3.PcmDecoder.java
public class PcmDecoder {
/**
* 初始化解码器
* @param mediaFilePath 原始媒体文件路径
* @param pcmFilePath 目标pcm文件生成路径
* @return
*/
public native int init(String mediaFilePath,String pcmFilePath);
/**
* 开始解码
*/
public native void decode();
/**
* 销毁解码器
*/
public native void destroy();
}
4.pcm_decoder.cpp
extern "C" {
/**
* 初始化
* @param env
* @param obj
* @param mediaFilePath 待解码的媒体文件路径
* @param pcmFilePath 生成pcm文件的路径
* @return
*/
PCMDecoderController *controller;
JNIEXPORT jint JNICALL
Java_com_yw_ffmpeg_decoder_PcmDecoder_init(JNIEnv *env, jobject obj, jstring mediaFilePath,
jstring pcmFilePath) {
//将jni字符串转换为c能识别的字符数组
const char *pcmPath = env->GetStringUTFChars(pcmFilePath, NULL);
const char *mediaPath = env->GetStringUTFChars(mediaFilePath, NULL);
controller = new PCMDecoderController();
controller->init(mediaPath, pcmPath);
//销毁字符串
env->ReleaseStringUTFChars(mediaFilePath, mediaPath);
env->ReleaseStringUTFChars(pcmFilePath, pcmPath);
return 0;
}
/**
* 解码
* @param env
* @param obj
*/
JNIEXPORT void JNICALL Java_com_yw_ffmpeg_decoder_PcmDecoder_decode(JNIEnv *env, jobject obj) {
if (controller) {
controller->decode();
}
}
/**
* 销毁
* @param env
* @param obj
*/
JNIEXPORT void JNICALL Java_com_yw_ffmpeg_decoder_PcmDecoder_destroy(JNIEnv *env, jobject obj) {
if (controller) {
controller->destroy();
delete controller;
controller = NULL;
}
}
}
5.pcm_deocder_controller.cpp
#include "pcm_decoder_controller.h"
PCMDecoderController::PCMDecoderController() {
realDecoder = NULL;
pcmFile = NULL;
}
PCMDecoderController::~PCMDecoderController() {
}
void PCMDecoderController::init(const char *orgFilePath, const char *pcmFilePath) {
//初始化decoder
RealDecoder *tempDecoder = new RealDecoder();
int metaData[2];
tempDecoder->getMusicMeta(orgFilePath, metaData);
delete tempDecoder;
//初始化采样率
sampleRate = metaData[0];
int byteCountPreSec = sampleRate * 2 * 16 / 8;
packetBufferSize = (int) ((byteCountPreSec / 2) * 0.2);
realDecoder = new RealDecoder();
realDecoder->init(orgFilePath, packetBufferSize);
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "init PCMDecoderController success");
pcmFile = fopen(pcmFilePath, "wb+");
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "init pcmFile success");
}
void PCMDecoderController::decode() {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "start decode audio data");
while (true) {
AudioPacket *packet = realDecoder->decodePacket();
if (-1 == packet->size) {
break;
}
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "write buffer to file");
fwrite(packet->buffer, sizeof(short), packet->size, pcmFile);
}
}
void PCMDecoderController::destroy() {
if (NULL != realDecoder) {
realDecoder->destroy();
delete realDecoder;
realDecoder = NULL;
}
if (NULL != pcmFile) {
fclose(pcmFile);
pcmFile = NULL;
}
}
6.pcm_real_decoder.cpp
#include "pcm_real_decoder.h"
RealDecoder::RealDecoder() {
this->seek_seconds = 0.0f;
this->seek_req = false;
this->seek_resp = false;
mediaFilePath = NULL;
}
RealDecoder::~RealDecoder() {
if (NULL != mediaFilePath) {
delete[] mediaFilePath;
mediaFilePath = NULL;
}
}
int RealDecoder::getMusicMeta(const char *fileString, int *metaData) {
init(fileString);
int sampleRate = avCodecContext->sample_rate;
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "%d", "sampleRate:", sampleRate);
int bitRate = avCodecContext->bit_rate;
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "%d", "bitRate:", bitRate);
destroy();
metaData[0] = sampleRate;
metaData[1] = bitRate;
return 0;
}
void RealDecoder::init(const char *fileString, int packetBufferSizeParam) {
init(fileString);
packetBufferSize = packetBufferSizeParam;
}
int RealDecoder::init(const char *audioFile) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "init:");
audioBuffer = NULL;
position = -1.0f;
audioBufferCursor = 0;
audioBufferSize = 0;
swrContext = NULL;
swrBuffer = NULL;
swrBufferSize = 0;
seek_success_read_frame_success = true;
isNeedFirstFrameCorrectFlag = true;
firstFrameCorrectionInSecs = 0.0f;
av_register_all();
avFormatContext = avformat_alloc_context();
//打开输入文件
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "Open Input File");
if (NULL == mediaFilePath) {
int length = strlen(audioFile);
mediaFilePath = new char[length + 1];
//由于最后一个是' ' 所以memset的长度要设置为length+1
//作用是在一段内存块中填充某个给定的值，它对较大的结构体或数组进行清零操作的一种最快方法。
memset(mediaFilePath, 0, length + 1);
memcpy(mediaFilePath, audioFile, length + 1);
}
int result = avformat_open_input(&avFormatContext, audioFile, NULL, NULL);
if (result != 0) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "Open Input File Fail");
return -1;
} else {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "Open Input File success");
}
avFormatContext->max_analyze_duration = 50000;
//检查文件中的流信息
result = avformat_find_stream_info(avFormatContext, NULL);
if (result < 0) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "find stream Fail");
return -1;
} else {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "find stream success");
}
stream_index = av_find_best_stream(avFormatContext, AVMEDIA_TYPE_AUDIO, -1, -1, NULL, 0);
if (stream_index == -1) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "find audio fail");
return -1;
}
//获取音频流
AVStream *audioStream = avFormatContext->streams[stream_index];
if (audioStream->time_base.den && audioStream->time_base.num) {
time_base = av_q2d(audioStream->time_base);
} else if (audioStream->codec->time_base.den && audioStream->codec->time_base.num) {
time_base = av_q2d(audioStream->codec->time_base);
}
//获取音频流解码器上下文
avCodecContext = audioStream->codec;
//根据加码器上下文找到解码器
AVCodec *avCodec = avcodec_find_decoder(avCodecContext->codec_id);
if (avCodec == NULL) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "find decode fail");
return -1;
} else {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "find decode success");
}
//打开解码器
result = avcodec_open2(avCodecContext, avCodec, NULL);
if (result < 0) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "open decode fail");
return -1;
} else {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "open decode success");
}
//判断是否需要resampler,重采样
if (!audioCodecIsSupported()) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder",
"because of audio Codec Is Not Supported so we will init swresampler...");
/**
* 初始化resampler
* @param s Swr context, can be NULL
* @param out_ch_layout output channel layout (AV_CH_LAYOUT_*)
* @param out_sample_fmt output sample format (AV_SAMPLE_FMT_*).
* @param out_sample_rate output sample rate (frequency in Hz)
* @param in_ch_layout input channel layout (AV_CH_LAYOUT_*)
* @param in_sample_fmt input sample format (AV_SAMPLE_FMT_*).
* @param in_sample_rate input sample rate (frequency in Hz)
* @param log_offset logging level offset
* @param log_ctx parent logging context, can be NULL
*/
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "start init swrContext");
swrContext = swr_alloc_set_opts(NULL, av_get_default_channel_layout(OUT_PUT_CHANNELS),
AV_SAMPLE_FMT_S16, avCodecContext->sample_rate,
av_get_default_channel_layout(avCodecContext->channels),
avCodecContext->sample_fmt, avCodecContext->sample_rate, 0,
NULL);
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "end init swrContext");
if (!swrContext || swr_init(swrContext)) {
if (swrContext)
swr_free(&swrContext);
avcodec_close(avCodecContext);
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "init resampler failed...");
return -1;
}
}
//初始化AVFrame
audioFrame = av_frame_alloc();
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "open create audioFrame success");
return -1;
}
/**
* 检测音频编码是否支持S16
* @return
*/
bool RealDecoder::audioCodecIsSupported() {
if (avCodecContext->sample_fmt == AV_SAMPLE_FMT_S16) {
return true;
}
return false;
}
/**
* 解码Packet
* @return
*/
AudioPacket *RealDecoder::decodePacket() {
short *samples = new short[packetBufferSize];
int stereoSampleSize = readSamples(samples, packetBufferSize);
AudioPacket *samplePacket = new AudioPacket();
if (stereoSampleSize > 0) {
//构造成一个packet
samplePacket->buffer = samples;
samplePacket->size = stereoSampleSize;
/** 这里由于每一个packet的大小不一样有可能是200ms 但是这样子position就有可能不准确了 **/
samplePacket->position = position;
} else {
samplePacket->size = -1;
}
return samplePacket;
}
int RealDecoder::readSamples(short *samples, int size) {
if (seek_req) {
audioBufferCursor = audioBufferSize;
}
int sampleSize = size;
while (size > 0) {
if (audioBufferCursor < audioBufferSize) {
int audioBufferDataSize = audioBufferSize - audioBufferCursor;
int copySize = MIN(size, audioBufferDataSize);
memcpy(samples + (sampleSize - size), audioBuffer + audioBufferCursor, copySize * 2);
size -= copySize;
audioBufferCursor += copySize;
} else {
if (readFrame() < 0) {
break;
}
}
// LOGI("size is %d", size);
}
int fillSize = sampleSize - size;
if (fillSize == 0) {
return -1;
}
return fillSize;
}
void RealDecoder::seek_frame() {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder",
"
seek frame firstFrameCorrectionInSecs is %.6f, seek_seconds=%f, position=%f
",
firstFrameCorrectionInSecs, seek_seconds, position);
float targetPosition = seek_seconds;
float currentPosition = position;
float frameDuration = duration;
if (targetPosition < currentPosition) {
this->destroy();
this->init(mediaFilePath);
//这里GT的测试样本会差距25ms 不会累加
currentPosition = 0.0;
}
int readFrameCode = -1;
while (true) {
av_init_packet(&packet);
readFrameCode = av_read_frame(avFormatContext, &packet);
if (readFrameCode >= 0) {
currentPosition += frameDuration;
if (currentPosition >= targetPosition) {
break;
}
}
// LOGI("currentPosition is %.3f", currentPosition);
av_free_packet(&packet);
}
seek_resp = true;
seek_req = false;
seek_success_read_frame_success = false;
}
/**
* 读取视频中的每一帧
* @return
*/
int RealDecoder::readFrame() {
if (seek_req) {
this->seek_frame();
}
int ret = 1;
av_init_packet(&packet);
int gotFrame = -1;
int readFrameCode = -1;
while (true) {
readFrameCode = av_read_frame(avFormatContext, &packet);
if (readFrameCode >= 0) {
if (packet.stream_index == stream_index) {//说明是音频数据
int len = avcodec_decode_audio4(avCodecContext, audioFrame, &gotFrame, &packet);
if (len < 0) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder",
"decode audio error, skip packet");
}
if (gotFrame) {
int numChannels = OUT_PUT_CHANNELS;
int numFrames = 0;
void *audioData;
if (swrContext) {
const int ratio = 2;
const int bufSize = av_samples_get_buffer_size(NULL,
numChannels,
audioFrame->nb_samples *
ratio,
AV_SAMPLE_FMT_S16, 1);
if (!swrBuffer || swrBufferSize < bufSize) {
swrBufferSize = bufSize;
swrBuffer = realloc(swrBuffer, swrBufferSize);
}
byte *outbuf[2] = {(byte *) swrBuffer, NULL};
numFrames = swr_convert(swrContext, outbuf,
audioFrame->nb_samples * ratio,
(const uint8_t **) audioFrame->data,
audioFrame->nb_samples);
if (numFrames < 0) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder",
"fail resample audio");
ret = -1;
break;
}
audioData = swrBuffer;
} else {
if (avCodecContext->sample_fmt != AV_SAMPLE_FMT_S16) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder",
"bucheck, audio format is invalid");
ret = -1;
break;
}
audioData = audioFrame->data[0];
numFrames = audioFrame->nb_samples;
}
if (isNeedFirstFrameCorrectFlag && position >= 0) {
float expectedPosition = position + duration;
float actualPosition =
av_frame_get_best_effort_timestamp(audioFrame) * time_base;
firstFrameCorrectionInSecs = actualPosition - expectedPosition;
isNeedFirstFrameCorrectFlag = false;
}
duration = av_frame_get_pkt_duration(audioFrame) * time_base;
position = av_frame_get_best_effort_timestamp(audioFrame) * time_base -
firstFrameCorrectionInSecs;
if (!seek_success_read_frame_success) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "position is %.6f",
position);
actualSeekPosition = position;
seek_success_read_frame_success = true;
}
audioBufferSize = numFrames * numChannels;
// LOGI("
duration is %.6f position is %.6f audioBufferSize is %d
", duration, position, audioBufferSize);
audioBuffer = (short *) audioData;
audioBufferCursor = 0;
break;
}
}
} else {
ret = -1;
break;
}
}
av_free_packet(&packet);
return ret;
}
void RealDecoder::destroy() {
// LOGI("start destroy!!!");
if (NULL != swrBuffer) {
free(swrBuffer);
swrBuffer = NULL;
swrBufferSize = 0;
}
if (NULL != swrContext) {
swr_free(&swrContext);
swrContext = NULL;
}
if (NULL != audioFrame) {
av_free(audioFrame);
audioFrame = NULL;
}
if (NULL != avCodecContext) {
avcodec_close(avCodecContext);
avCodecContext = NULL;
}
if (NULL != avFormatContext) {
__android_log_print(ANDROID_LOG_ERROR, "RealDecoder", "leave LiveReceiver::destory");
avformat_close_input(&avFormatContext);
avFormatContext = NULL;
}
// LOGI("end destroy!!!");
}
三、源代码下载