Android Exoplayer多路不同时长音视频混合播放

在上一篇Android Exoplayer 实现多个音视频文件混合播放以及音轨切换中我们提到一个问题，如果视频和音频时长不一致，特别是想混合多个音频和多个视频时就会出问题，无法播放。报错如下：

E/ExoPlayerImplInternal(11191): Playback error
E/ExoPlayerImplInternal(11191):   com.google.android.exoplayer2.ExoPlaybackException: Source error
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.ExoPlayerImplInternal.handleIoException(ExoPlayerImplInternal.java:684)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:660)
E/ExoPlayerImplInternal(11191):       at android.os.Handler.dispatchMessage(Handler.java:98)
E/ExoPlayerImplInternal(11191):       at android.os.Looper.loop(Looper.java:136)
E/ExoPlayerImplInternal(11191):       at android.os.HandlerThread.run(HandlerThread.java:61)
E/ExoPlayerImplInternal(11191):   Caused by: com.google.android.exoplayer2.source.MergingMediaSource$IllegalMergeException
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.MergingMediaSource.onChildSourceInfoRefreshed(MergingMediaSource.java:252)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.MergingMediaSource.onChildSourceInfoRefreshed(MergingMediaSource.java:52)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.CompositeMediaSource.lambda$prepareChildSource$0$com-google-android-exoplayer2-source-CompositeMediaSource(CompositeMediaSource.java:120)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.CompositeMediaSource$$ExternalSyntheticLambda0.onSourceInfoRefreshed(D8$$SyntheticClass:0)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.BaseMediaSource.refreshSourceInfo(BaseMediaSource.java:94)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.ConcatenatingMediaSource.updateTimelineAndScheduleOnCompletionActions(ConcatenatingMediaSource.java:746)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.ConcatenatingMediaSource.handleMessage(ConcatenatingMediaSource.java:716)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.ConcatenatingMediaSource.$r8$lambda$xvlxaabNVihM68DRWdn_WPenrXk(ConcatenatingMediaSource.java)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.ConcatenatingMediaSource$$ExternalSyntheticLambda0.handleMessage(D8$$SyntheticClass:0)
E/ExoPlayerImplInternal(11191):       ... 3 more

这个主要是播放时长不一致，无法同步时序导致。

使用场景：比如K歌应用中，没有原版MV，只有音频文件，想给音频配一个背景视频或多个混合视频当MV，但视频均是风景短片，时长与音频不一致。当用Exoplayer进行混合播放时，我们希望以音频时长为准，在音频播放完成前，视频循环播放。

一直没有找到很好的方法解决，最后采取了一个笨办法，启用两个播放器，一个专门播放视频，一个专门播放音频，这样视频任意混合或循环播放都与音频互不干扰，就可用规避时序错乱问题。

以下为实现样例：

 private ExoPlayer mExoPlayer,mExoPlayer2;//音频播放器mExoPlayer = new ExoPlayer.Builder(context, renderersFactory).setTrackSelector(trackSelector).build();// 检查音频配置AudioAttributes audioAttributes = new AudioAttributes.Builder().setUsage(C.USAGE_MEDIA).setContentType(C.AUDIO_CONTENT_TYPE_MUSIC).build();mExoPlayer.setAudioAttributes(audioAttributes, true);//视频播放器mExoPlayer2 = new ExoPlayer.Builder(context, new DefaultMediaSourceFactory(context)).setVideoScalingMode(MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT).build();DefaultMediaSourceFactory mediaSourceFactory = new DefaultMediaSourceFactory(dataSourceFactory);// 创建两个视频的 MediaSourceMediaSource video1Source = mediaSourceFactory.createMediaSource(MediaItem.fromUri("asset://android_asset/01.mp4"));MediaSource video2Source = mediaSourceFactory.createMediaSource(MediaItem.fromUri("asset://android_asset/02.mp4"));ConcatenatingMediaSource concatenatingMediaSource = new ConcatenatingMediaSource(video1Source,video2Source);LoopingMediaSource loopingMediaSource = new LoopingMediaSource(concatenatingMediaSource);// 合并两个音频源MediaSource audio1Source = mediaSourceFactory.createMediaSource(MediaItem.fromUri("asset://android_asset/audio/ori.mp2"));MediaSource audio2Source = mediaSourceFactory.createMediaSource(MediaItem.fromUri("asset://android_asset/audio/acc.mp2"));MergingMediaSource audioMerged = new MergingMediaSource(audio1Source, audio2Source);mExoPlayer2.setMediaSource(loopingMediaSource);// mExoPlayer2.setRepeatMode(Player.REPEAT_MODE_ONE);// 添加到 ExoPlayermExoPlayer.setMediaSource(audioMerged);

需要注意的是两个播放器要保持状态同步，以播放进度音频播放器为准。

   @Overridepublic void prepareAsync() throws IllegalStateException {mExoPlayer.prepare();mExoPlayer2.prepare();}@Overridepublic void start() throws IllegalStateException {mExoPlayer.setPlayWhenReady(true);mExoPlayer2.setPlayWhenReady(true);//  getCurrentPostion();}@Overridepublic void stop() throws IllegalStateException {mExoPlayer.stop();mExoPlayer2.stop();}@Overridepublic void pause() throws IllegalStateException {mExoPlayer.setPlayWhenReady(false);mExoPlayer2.setPlayWhenReady(false);}@Overridepublic void setSpeed(float speed) {PlaybackParameters parameters = new PlaybackParameters(speed);mExoPlayer.setPlaybackParameters(parameters);}@Overridepublic long getCurrentPosition() {if (mExoPlayer == null)return 0;return mExoPlayer.getCurrentPosition();}

还有就是切换音轨的时候需要注意，由于音视频分开处理了，切换音轨的时候只处理音频播放器即可，切换分辨率的时候只处理视频播放器即可，这时媒体轨道数会比音视频混合一起的情况要少一些，因为只有视频或只有音频轨道，切换时轨道索引值参数肯定要小些了。

比如上面样例音频播放器只有两个音频轨道，所以切换音轨时，索引只有0或1.

//原唱
TrackGroup selectedGroup = currentTrackGroups.get(0);
//伴奏
//TrackGroup selectedGroup = currentTrackGroups.get(1);// 应用新音轨mExoPlayer.setTrackSelectionParameters(mExoPlayer.getTrackSelectionParameters().buildUpon().setOverrideForType(new TrackSelectionOverride(selectedGroup, 0))  //需要切换到的音轨索引.build());

这样就可以迂回解决多路不同时长音视频混合流播放问题。大佬们有其他更好解决办法的话欢饮留言交流。