Raw Data

The Video SDK provides you with an option to access real-time raw audio and video data of a session. After accessing the raw data, you can process it and apply additional effects to it to enhance the session experience.

This section will provide you with information on how to enable and gain access to the raw data of a session.

Specify raw data memory mode

In order to utilize raw data of any type, you must first select the memory mode you wish to use. The SDK supports heap-based and stack-based memory modes.

Stack-Based Memory

  • Variables are allocated automatically and deallocated after the data leaves scope.
  • Variables are not accessible from or transferable to other threads.
  • Typically has faster access than heap-based memory allocation.
  • Memory space is managed by the CPU and will not become fragmented.
  • Variables cannot be resized.

Heap-Based Memory

  • Variables are allocated and deallocated manually. You must allocate and free variables to avoid memory leaks.
  • Variables can be accessed globally.
  • Has relatively slower access than stack-based memory allocation.
  • Has no guarantee on the efficiency of memory space and can become fragmented.
  • Variables can be resized.

For more information on these types of memory allocation, refer to Stack-based memory allocation and Heap-based dynamic memory allocation guides.

After determining which memory mode is right for you, you must specify it when the SDK is initialized. Note that this must be done for audio and video individually. The two types of memory allocation are defined with the following enum values:

Stack-based Memory

ZMVideoSDKRawDataMemoryMode_Stack

Heap-based Memory

ZMVideoSDKRawDataMemoryMode_Heap

To specify a raw data memory mode, provide one of these enums cases to the ZMVideoSDKInitParams during SDK initialization.


ZMVideoSDKInitParams *initParams = [[ZMVideoSDKInitParams alloc] init];
// Set audio memory mode to heap.
initParams.audioRawdataMemoryMode = ZMVideoSDKRawDataMemoryMode_Heap;
// Set video memory mode to heap.
initParams.videoRawdataMemoryMode = ZMVideoSDKRawDataMemoryMode_Heap;
// Set share memory mode to heap.
initParams.shareRawdataMemoryMode = ZMVideoSDKRawDataMemoryMode_Heap;

let initParams = ZMVideoSDKInitParams()
// Set audio memory mode to heap.
initParams.audioRawdataMemoryMode = ZMVideoSDKRawDataMemoryMode_Heap
// Set video memory mode to heap.
initParams.videoRawdataMemoryMode = ZMVideoSDKRawDataMemoryMode_Heap
// Set share memory mode to heap.
initParams.shareRawdataMemoryMode = ZMVideoSDKRawDataMemoryMode_Heap

Set raw audio memory mode


ZMVideoSDKInitParams *params = [[ZMVideoSDKInitParams alloc] init];
params.audioRawDataMemoryMode = ZMVideoSDKRawDataMemoryMode_Heap;

let initParams = ZMVideoSDKInitParams()
initParams.audioRawDataMemoryMode = ZMVideoSDKRawDataMemoryMode_Heap

Receive raw audio data

You can access mixed and per-user raw audio data. Raw data will default to stack-based memory if you do not specify a memory mode.


- (void) onMixedAudioRawDataReceived:(ZMVideoSDKAudioRawData *)data {
    // Access audio raw data for the whole session here.
}
- (void) onSharedAudioRawDataReceived:(ZMVideoSDKAudioRawData *)data {
    // Access share audio raw data here.
}
- (void) onOneWayAudioRawDataReceived:(ZMVideoSDKAudioRawData *)data user:(ZMVideoSDKUser *)user {
    // Access per-user audio raw data here.
}

func onMixedAudioRawDataReceived(_ data: ZMVideoSDKAudioRawData!) {
    // Access audio raw data for the whole session here.
}
func onSharedAudioRawDataReceived(_ data: ZMVideoSDKAudioRawData!) {
    // Access share audio raw data here.
}
func onOneWayAudioRawDataReceived(_ data: ZMVideoSDKAudioRawData!, user: ZMVideoSDKUser!) {
    // Access per-user audio raw data here.
}

Send raw audio data


- (void) onMicInitialize:(ZMVideoSDKAudioSender *)rawdataSender {
    // Store sender for later use
    self.sender = rawdataSender;
}
[sender send:data dataLength:dataLength sampleRate:sampleRate];

func onMicInitialize(_ rawdataSender: ZMVideoSDKAudioSender) {
    // Store sender for later use.
    self.sender = rawdataSender
}
sender?.send(data, dataLength: dataLength, sampleRate: sampleRate)

Send raw audio data in class extending ZMVideoSDKVirtualAudioMic:

Receive raw audio for virtual speaker


- (void) onVirtualSpeakerMixedAudioReceived:(ZMVideoSDKAudioRawData *)data {
    // Access audio raw data for the whole session here.
}
- (void) onVirtualSpeakerSharedAudioReceived:(ZMVideoSDKAudioRawData *)data {
    // Access share audio raw data here.
}
- (void) onVirtualSpeakerOneWayAudioReceived:(ZMVideoSDKAudioRawData *)data user:(ZMVideoSDKUser *)user {
    // Access per-user audio raw data here.
}

func onVirtualSpeakerMixedAudioReceived(_ rawdata: ZMVideoSDKAudioRawData) {
    // Access audio raw data for the whole session here.
}
func onVirtualSpeakerOneWayAudioReceived(_ rawdata: ZMVideoSDKAudioRawData, user: ZMVideoSDKUser) {
    // Access per-user audio raw data here.
}
func onVirtualSpeakerSharedAudioReceived(_ rawdata: ZMVideoSDKAudioRawData) {
    // Access share audio raw data here.
}

From within the callbacks, you can access the data buffer with rawData.getBuffer().

For more information on implementing your ZMVideoSDKDelegate, refer to Listen for callbacks.

Next Steps

Need help?

If you're looking for help, try Developer Support or our Developer Forum. Priority support is also available with Premier Developer Support plans.