Interface GLMediaPlayer
- All Superinterfaces:
TextureSequence
TextureSequence
state machine
using a multiplexed audio/video stream as it's source.
Audio maybe supported and played back internally or via an AudioSink
implementation.
Audio and video streams can be selected or muted via initStream(Uri, int, int, int)
using the appropriate stream id's.
Camera input can be selected using the CameraInputScheme
Uri.
StreamWorker Decoding Thread
Most of the stream processing is performed on the decoding thread, a.k.a. StreamWorker:
- Stream initialization triggered by
initStream(..)
- User gets notified whether the stream has been initialized or not viaattributesChanges(..)
. - Stream decoding - User gets notified of a new frame via
newFrameAvailable(...)
. - Caught exceptions on the decoding thread are delivered as
GLMediaPlayer.StreamException
s.
GLContext
, shared with the one passed to initGL(GL)
.
The shared GLContext
allows the decoding thread to push the video frame data directly into
the designated TextureSequence.TextureFrame
, later returned via getNextTexture(GL)
and used by the user.
Caught exceptions on StreamWorker are delivered as GLMediaPlayer.StreamException
s,
which either degrades the GLMediaPlayer.State
to GLMediaPlayer.State.Uninitialized
or GLMediaPlayer.State.Paused
.
An occurring GLMediaPlayer.StreamException
triggers a EVENT_CHANGE_ERR
event,
which can be listened to via GLMediaPlayer.GLMediaEventListener.attributesChanged(GLMediaPlayer, int, long)
.
An occurred GLMediaPlayer.StreamException
can be read via getStreamException()
.
GLMediaPlayer Lifecycle
Audio and video Stream IDs
value | request | get |
---|---|---|
STREAM_ID_NONE | mute | not available |
STREAM_ID_AUTO | auto | unspecified |
≥0 | specific stream | specific stream |
Current implementations (check each API doc link for details):
NullGLMediaPlayer
OMXGLMediaPlayer
FFMPEGMediaPlayer
AndroidGLMediaPlayerAPI14
Implementations of this interface must implement:
public static final boolean isAvailable();to be properly considered by
GLMediaPlayerFactory.create(ClassLoader, String)
and GLMediaPlayerFactory.createDefault()
.
Timestamp Accuracy
Timestamp type and value range has been chosen to suit embedded CPUs
and characteristics of audio and video streaming. See TimeFrameI
.
Audio and video synchronization
The class follows a passive A/V synchronization pattern.
Audio is being untouched, while getNextTexture(GL)
delivers a new video frame
only, if its timestamp is less than MAXIMUM_VIDEO_ASYNC
ahead of time.
If its timestamp is more than MAXIMUM_VIDEO_ASYNC
ahead of time,
the previous frame is returned.
If its timestamp is more than MAXIMUM_VIDEO_ASYNC
after time,
the frame is dropped and the next frame is being fetched.
https://en.wikipedia.org/wiki/Audio_to_video_synchronization
d_av = v_pts - a_pts;
Recommendation of audio/video pts time lead/lag at production:
- Overall: +40ms and -60ms audio ahead video / audio after video
- Each stage: +5ms and -15ms. audio ahead video / audio after video
Recommendation of av pts time lead/lag at presentation:
- TV: +15ms and -45ms. audio ahead video / audio after video.
- Film: +22ms and -22ms. audio ahead video / audio after video.
Test Streams
Big Buck Bunny 24f 16:9 | ||||
---|---|---|---|---|
Big Buck Bunny | 320p | h264 | aac 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/BigBuckBunny_320x180.mp4 |
Big Buck Bunny | 240p | h264 | aac 48000Hz 2 chan | http://archive.org/download/BigBuckBunny_328/BigBuckBunny_512kb.mp4 |
Big Buck Bunny | 720p | mpeg4 | ac3 48000Hz 5.1 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_surround.avi |
Big Buck Bunny | 720p | msmpeg4v2 | mp3 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.avi |
Big Buck Bunny | 720p | theora | vorbis 48000Hz 2 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_720p_stereo.ogg |
Big Buck Bunny | 1080p | mpeg4 | ac3 48000Hz 5.1 chan | http://download.blender.org/peach/bigbuckbunny_movies/big_buck_bunny_1080p_surround.avi |
WebM/Matroska (vp8/vorbis) | ||||
Big Buck Bunny Trailer | 640p | vp8 | vorbis 44100Hz 1 chan | http://video.webmfiles.org/big-buck-bunny_trailer.webm |
Elephants Dream | 540p | vp8 | vorbis 44100Hz 1 chan | http://video.webmfiles.org/elephants-dream.webm |
You Tube http/rtsp | ||||
Sintel | http://www.youtube.com/watch?v=eRsGyueVLvQ | rtsp://v3.cache1.c.youtube.com/CiILENy73wIaGQn0LpXnygYbeRMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp | ||
Audio/Video Sync | ||||
Five-minute-sync-test1080p | https://www.youtube.com/watch?v=szoOsG9137U | rtsp://v7.cache8.c.youtube.com/CiILENy73wIaGQm133VvsA46sxMYDSANFEgGUgZ2aWRlb3MM/0/0/0/video.3gp | ||
Audio-Video-Sync-Test-Calibration-23.98fps-24fps | https://www.youtube.com/watch?v=cGgf_dbDMsw | |||
sound_in_sync_test | https://www.youtube.com/watch?v=O-zIZkhXNLE | |||
title | url1 | url2 |
Since 2.3.0 this interface uses Uri
instead of URI
.
-
Nested Class Summary
Nested ClassesModifier and TypeInterfaceDescriptionstatic interface
static enum
See Lifecycle.static class
A StreamException encapsulates a caught exception in the decoder thread, a.k.a StreamWorker, see See StreamWorker Error Handling.Nested classes/interfaces inherited from interface com.jogamp.opengl.util.texture.TextureSequence
TextureSequence.TexSeqEventListener<T extends TextureSequence>, TextureSequence.TextureFrame
-
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final com.jogamp.common.net.Uri.Encoded
Uri scheme
name for camera input.static final String
Camera property "height".static final String
Camera property "rate".static final String
Camera property "size", size as string, e.g.static final String
Camera property "width".static final boolean
static final boolean
static final int
Maximum video frame async of 22 milliseconds.static final int
Constant -1 for auto or unspecified.static final int
Constant -2 for mute or not available.static final int
Default texture count, value 4.static final int
Minimum texture count, value 1.Fields inherited from interface com.jogamp.opengl.util.texture.TextureSequence
sampler2D, samplerExternalOES
-
Method Summary
Modifier and TypeMethodDescriptionvoid
Adds aGLMediaPlayer.GLMediaEventListener
to this player.attachObject
(String name, Object obj) Attaches the user object for the given name.Releases the GL, stream and other resources, includingattached user objects
.detachObject
(String name) Detaches the user object for the given name.int
getAID()
Return the audio stream id, see audio and video Stream IDs.getAttachedObject
(String name) Returns the attached user object for the given name.int
Warning: Optional information, may not be supported by implementation.Warning: Optional information, may not be supported by implementation.int
Warning: Optional information, may not be supported by implementation.int
If implementation uses aAudioSink
, it's instance will be returned.float
Returns the audio volume.int
int
Return allGLMediaPlayer.GLMediaEventListener
of this player.float
Warning: Optional information, may not be supported by implementation.int
Returns the height of the video.Returns the last updated texture.getNextTexture
(GL gl) Returns the next texture to be rendered.Returns a string represantation of this player's performance values.float
Returns the playback speed.int
getState()
See Lifecycle.long
Warning: Optional information, may not be supported by implementation.Returns theGLMediaPlayer.StreamException
caught in the decoder thread, ornull
if none occured.int
com.jogamp.common.net.Uri
getUri()
Return the stream location, as set byinitStream(Uri, int, int, int)
.int
getVID()
Return the video stream id, see audio and video Stream IDs.int
Warning: Optional information, may not be supported by implementation.Warning: Optional information, may not be supported by implementation.int
Warning: Optional information, may not be supported by implementation.int
int
getWidth()
Returns the width of the video.void
Initializes OpenGL related resources.void
initStream
(com.jogamp.common.net.Uri streamLoc, int vid, int aid, int textureCount) Issues asynchronous stream initialization.boolean
Returnstrue
if the video frame is oriented in OpenGL's coordinate system, origin at bottom left.pause
(boolean flush) Pauses the StreamWorker decoding thread.play()
Starts or resumes the StreamWorker decoding thread.void
Removes aGLMediaPlayer.GLMediaEventListener
to this player.int
seek
(int msec) Seeks to the new absolute position.boolean
setAudioVolume
(float v) Sets the audio volume, [0f..1f].boolean
setPlaySpeed
(float rate) Sets the playback speed.void
setTextureMinMagFilter
(int[] minMagFilter) Sets the texture min-mag filter, defaults toGL.GL_NEAREST
.void
setTextureUnit
(int u) Sets the texture unit.void
setTextureWrapST
(int[] wrapST) Sets the texture min-mag filter, defaults toGL.GL_CLAMP_TO_EDGE
.toString()
Returns a string represantation of this player, incl.Methods inherited from interface com.jogamp.opengl.util.texture.TextureSequence
getRequiredExtensionsShaderStub, getTextureFragmentShaderHashCode, getTextureLookupFragmentShaderImpl, getTextureLookupFunctionName, getTextureMinMagFilter, getTextureSampler2DType, getTextureTarget, getTextureUnit, getTextureWrapST, isTextureAvailable
-
Field Details
-
DEBUG
static final boolean DEBUG -
DEBUG_NATIVE
static final boolean DEBUG_NATIVE -
TEXTURE_COUNT_DEFAULT
static final int TEXTURE_COUNT_DEFAULTDefault texture count, value 4.- See Also:
-
TEXTURE_COUNT_MIN
static final int TEXTURE_COUNT_MINMinimum texture count, value 1. Using the minimum texture count disables multi-threaded decoding.- See Also:
-
STREAM_ID_NONE
static final int STREAM_ID_NONEConstant -2 for mute or not available. See Audio and video Stream IDs.- See Also:
-
STREAM_ID_AUTO
static final int STREAM_ID_AUTOConstant -1 for auto or unspecified. See Audio and video Stream IDs.- See Also:
-
CameraInputScheme
static final com.jogamp.common.net.Uri.Encoded CameraInputSchemeUri scheme
name for camera input. E.g.camera:/0
for the 1st camera device.The
Uri path
is being used to identify the camera (<id>), where the root fwd-slash is being cut-off.The <id> is usually an integer value indexing the camera ranging from [0..max-number].
The <somewhere> is usually empty, since it would imply a networking camera protocol.
The
Uri query
is used to pass options to the camera using ; as the separator. The latter avoids trouble w/ escaping.camera:/<id> camera:/<id>?width=640;height=480;rate=15 camera:/<id>?size=640x480;rate=15 camera://<somewhere>/<id> camera://<somewhere>/<id>?width=640;height=480;rate=15 camera://<somewhere>/<id>?size=640x480;rate=15 camera:///<id>?width=640;height=480;rate=15 camera:///<id>?size=640x480;rate=15
Uri: [scheme:][//authority][path][?query][#fragment] w/ authority: [user-info@]host[:port] Note: 'path' starts w/ fwd slash
-
CameraPropSizeS
Camera property "size", size as string, e.g.1280x720
,hd720
. May not be supported on all platforms. SeeCameraInputScheme
.- See Also:
-
CameraPropWidth
Camera property "width". SeeCameraInputScheme
.- See Also:
-
CameraPropHeight
Camera property "height". SeeCameraInputScheme
.- See Also:
-
CameraPropRate
Camera property "rate". SeeCameraInputScheme
.- See Also:
-
MAXIMUM_VIDEO_ASYNC
static final int MAXIMUM_VIDEO_ASYNCMaximum video frame async of 22 milliseconds.- See Also:
-
-
Method Details
-
getTextureCount
int getTextureCount() -
setTextureUnit
void setTextureUnit(int u) Sets the texture unit. Defaults to 0. -
setTextureMinMagFilter
void setTextureMinMagFilter(int[] minMagFilter) Sets the texture min-mag filter, defaults toGL.GL_NEAREST
. -
setTextureWrapST
void setTextureWrapST(int[] wrapST) Sets the texture min-mag filter, defaults toGL.GL_CLAMP_TO_EDGE
. -
initStream
void initStream(com.jogamp.common.net.Uri streamLoc, int vid, int aid, int textureCount) throws IllegalStateException, IllegalArgumentException Issues asynchronous stream initialization.Lifecycle:
GLMediaPlayer.State.Uninitialized
->GLMediaPlayer.State.Initialized
1 orGLMediaPlayer.State.Uninitialized
GLMediaPlayer.State.Initialized
is reached asynchronous, i.e. user gets notified viaattributesChanges(..)
.A possible caught asynchronous
GLMediaPlayer.StreamException
while initializing the stream off-thread will be thrown atinitGL(GL)
.Muted audio can be achieved by passing
STREAM_ID_NONE
toaid
.Muted video can be achieved by passing
STREAM_ID_NONE
tovid
, in which casetextureCount
is ignored as well as the passed GL object of the subsequentinitGL(GL)
call.- Parameters:
streamLoc
- the stream locationvid
- video stream id, see audio and video Stream IDsaid
- video stream id, see audio and video Stream IDstextureCount
- desired number of buffered textures to be decoded off-thread, will be validated by implementation. The minimum value isTEXTURE_COUNT_DEFAULT
. Ignored if video is muted.- Throws:
IllegalStateException
- if not invoked inGLMediaPlayer.State.Uninitialized
IllegalArgumentException
- if arguments are invalid- Since:
- 2.3.0
-
getStreamException
GLMediaPlayer.StreamException getStreamException()Returns theGLMediaPlayer.StreamException
caught in the decoder thread, ornull
if none occured.Method clears the cached
GLMediaPlayer.StreamException
, hence an immediate subsequent call will returnnull
. -
initGL
Initializes OpenGL related resources.Lifecycle:
ArgumentGLMediaPlayer.State.Initialized
->GLMediaPlayer.State.Paused
orGLMediaPlayer.State.Initialized
gl
is ignored if video is muted, seeinitStream(Uri, int, int, int)
.- Parameters:
gl
- current GL object. Maybenull
, for audio only.- Throws:
IllegalStateException
- if not invoked inGLMediaPlayer.State.Initialized
.GLMediaPlayer.StreamException
- forwarded from the off-thread stream initializationGLException
- in case of difficulties to initialize the GL resources
-
getAudioSink
AudioSink getAudioSink()If implementation uses aAudioSink
, it's instance will be returned.The
AudioSink
instance is available afterinitStream(Uri, int, int, int)
, if used by implementation. -
destroy
Releases the GL, stream and other resources, includingattached user objects
. -
setPlaySpeed
boolean setPlaySpeed(float rate) Sets the playback speed.To simplify test, play speed is normalized, i.e.
1.0f
: ifMath.abs(1.0f - rate) < 0.01f
- Returns:
- true if successful, otherwise false, i.e. due to unsupported value range of implementation.
-
getPlaySpeed
float getPlaySpeed()Returns the playback speed. -
setAudioVolume
boolean setAudioVolume(float v) Sets the audio volume, [0f..1f].To simplify test, volume is normalized, i.e.
0.0f
: ifMath.abs(v) < 0.01f
1.0f
: ifMath.abs(1.0f - v) < 0.01f
- Returns:
- true if successful, otherwise false, i.e. due to unsupported value range of implementation.
-
getAudioVolume
float getAudioVolume()Returns the audio volume. -
play
GLMediaPlayer.State play()Starts or resumes the StreamWorker decoding thread.Lifecycle:
GLMediaPlayer.State.Paused
->GLMediaPlayer.State.Playing
-
pause
Pauses the StreamWorker decoding thread.Lifecycle:
GLMediaPlayer.State.Playing
->GLMediaPlayer.State.Paused
If a new frame is desired after the next
play()
call, e.g. to make a snapshot of a camera input stream,flush
shall be set totrue
.- Parameters:
flush
- iftrue
flushes the video and audio buffers, otherwise keep them intact.
-
seek
int seek(int msec) Seeks to the new absolute position. The StreamWorker decoding thread is paused while doing so and the A/V buffers are flushed.Allowed in state
GLMediaPlayer.State.Playing
andGLMediaPlayer.State.Paused
, otherwise ignored, see Lifecycle.- Parameters:
msec
- absolute desired time position in milliseconds- Returns:
- time current position in milliseconds, after seeking to the desired position
-
getState
GLMediaPlayer.State getState()See Lifecycle.- Returns:
- the current state, either
GLMediaPlayer.State.Uninitialized
,GLMediaPlayer.State.Initialized
,GLMediaPlayer.State.Playing
orGLMediaPlayer.State.Paused
-
getVID
int getVID()Return the video stream id, see audio and video Stream IDs. -
getAID
int getAID()Return the audio stream id, see audio and video Stream IDs. -
getDecodedFrameCount
int getDecodedFrameCount()- Returns:
- the current decoded frame count since
play()
andseek(int)
as increased bygetNextTexture(GL)
or the decoding thread.
-
getPresentedFrameCount
int getPresentedFrameCount()- Returns:
- the current presented frame count since
play()
andseek(int)
as increased bygetNextTexture(GL)
for new frames.
-
getVideoPTS
int getVideoPTS()- Returns:
- current video presentation timestamp (PTS) in milliseconds of
getLastTexture()
-
getAudioPTS
int getAudioPTS()- Returns:
- current audio presentation timestamp (PTS) in milliseconds.
-
getLastTexture
Returns the last updated texture.In case the instance is just initialized, it shall return a
Not blocking.TextureFrame
object with valid attributes. The texture content may be undefined until the first call ofTextureSequence.getNextTexture(GL)
.
- Specified by:
getLastTexture
in interfaceTextureSequence
- Throws:
IllegalStateException
- if not invoked inGLMediaPlayer.State.Paused
orGLMediaPlayer.State.Playing
-
getNextTexture
Returns the next texture to be rendered.Implementation shall return the next frame if available, may block if a next frame may arrive soon. Otherwise implementation shall return the last frame.
Shall return
null
in case no next or last frame is available.In case the current state is not
GLMediaPlayer.State.Playing
,getLastTexture()
is returned.- Specified by:
getNextTexture
in interfaceTextureSequence
- Throws:
IllegalStateException
- if not invoked inGLMediaPlayer.State.Paused
orGLMediaPlayer.State.Playing
- See Also:
-
addEventListener(GLMediaEventListener)
GLMediaEventListener#newFrameAvailable(GLMediaPlayer, TextureFrame, long)
-
getUri
com.jogamp.common.net.Uri getUri()Return the stream location, as set byinitStream(Uri, int, int, int)
.- Since:
- 2.3.0
-
getVideoCodec
String getVideoCodec()Warning: Optional information, may not be supported by implementation.- Returns:
- the code of the video stream, if available
-
getAudioCodec
String getAudioCodec()Warning: Optional information, may not be supported by implementation.- Returns:
- the code of the audio stream, if available
-
getVideoFrames
int getVideoFrames()Warning: Optional information, may not be supported by implementation.- Returns:
- the total number of video frames
-
getAudioFrames
int getAudioFrames()Warning: Optional information, may not be supported by implementation.- Returns:
- the total number of audio frames
-
getDuration
int getDuration()- Returns:
- total duration of stream in msec.
-
getStreamBitrate
long getStreamBitrate()Warning: Optional information, may not be supported by implementation.- Returns:
- the overall bitrate of the stream.
-
getVideoBitrate
int getVideoBitrate()Warning: Optional information, may not be supported by implementation.- Returns:
- video bitrate
-
getAudioBitrate
int getAudioBitrate()Warning: Optional information, may not be supported by implementation.- Returns:
- the audio bitrate
-
getFramerate
float getFramerate()Warning: Optional information, may not be supported by implementation.- Returns:
- the framerate of the video
-
isGLOriented
boolean isGLOriented()Returnstrue
if the video frame is oriented in OpenGL's coordinate system, origin at bottom left.Otherwise returns
false
, i.e. video frame is oriented origin at top left.false
is the default assumption for videos, but user shall not rely on.false
GL orientation leads toTexture.getMustFlipVertically()
==true
, as reflected by allTextureSequence.TextureFrame
'sTexture
s retrieved viagetLastTexture()
orgetNextTexture(GL)
. -
getWidth
int getWidth()Returns the width of the video. -
getHeight
int getHeight()Returns the height of the video. -
toString
String toString()Returns a string represantation of this player, incl. state and audio/video details. -
getPerfString
String getPerfString()Returns a string represantation of this player's performance values. -
addEventListener
Adds aGLMediaPlayer.GLMediaEventListener
to this player. -
removeEventListener
Removes aGLMediaPlayer.GLMediaEventListener
to this player. -
getEventListeners
GLMediaPlayer.GLMediaEventListener[] getEventListeners()Return allGLMediaPlayer.GLMediaEventListener
of this player. -
getAttachedObject
Returns the attached user object for the given name. -
attachObject
Attaches the user object for the given name. Returns the previously set object, may be null. -
detachObject
Detaches the user object for the given name. Returns the previously set object, may be null.
-