Audio Party-Chat Streaming in Unity using Agora

Hello, intrepid developers! In this tutorial you’ll set up an audio broadcast in a 3D Unity environment in Unity 2019.4.1 (LTS). This is essentially a video game environment where broadcasters can communicate with anyone in the same channel, and any audience member can listen in a game without video. On joining the scene, each player can choose via the UI to be either a broadcaster or an audience member, as well as the channel they want to join. Each broadcaster will have a gold material, and audience members will have the standard Viking aesthetic. You will use Agora callbacks to provide visual feedback for when each broadcaster is speaking.

Getting Started with Agora

This project builds on the agora-unity-audio-broadcasting demo.

Use agora-unity-audio-broadcasting as a starting template. Or create a project from scratch and import the Agora Video SDK and the Photon Viking Demo.

Development

Create Agora Engine

You now have a working Agora engine that you can access from your viking player objects.

But before moving on, in your viking scene add an EventSystem object to the scene via right-click in the Hierarchy > UI > Event System.

Broadcaster UI

Canvas Panel

Buttons

Toggle the BroadcasterSelectionPanel to the off state by removing the check mark next to its name in the Inspector.

Note: With multiple networked players in one scene, all of their UI panels will display unless toggled off. We want to display only the local user UI, and we will toggle it on through code.

Profile Selection Script

In ProfileSelection, make sure to include agora_gaming_rtc; and to replace

Public class ProfileSelection : MonoBehaviour with

Public class ProfileSelection : Photon.PunBehaviour. This gives you access to PUN services, and PunBehaviour gives you access to PUN callbacks you wouldn’t otherwise see.

First, you’ll create your attributes for the script:

Next, you’ll initialize the Agora engine. The engine itself is static, and a single instance is located in the scene. From each player, we look for the engine and access it. Because you are in a networked demo, there might be a slight delay in the synchronization from the player to the engine, so we build a timeout function to either retrieve the engine within 3 seconds or fail with an error:.

With a successful engine setup, the BroadcasterSelectionPanel will display, with buttons and an input field that have no functionality when clicked. Create a function called ButtonSetBroadcastState(bool isNewStateBroadcaster).

Note: When I create functions that are specifically intended to be assigned to buttons, I like to prefix them with “Button” so you can easily find them in the Inspector.

This project is meant for reference purposes and development environments, it is not intended for production environments. Token authentication is recommended for all RTE apps running in production environments. For more information about token based authentication within the Agora platform please refer to this guide: https://bit.ly/3sNiFRs

Next, create a function called TurnVikingGold. This function will send a “Remote Procedure Call” (RPC) across the network to change the necessary viking gold if the user is a broadcaster:

Note the [PunRPC] attribute, without it the function will be called only locally. That is, it will change your viking on your machine gold, but when you look at another client who is in the game with you, you won’t be gold on their screen!

Last, we will add one more function that synchronizes broadcaster/audience states as people start to join the channel. If someone is a broadcaster, but another person joins after they activate that status, the newly joined person won’t see the broadcaster’s gold material.

This is where inheriting from Photon.PunBehaviour is crucial, because without it this Photon event isn’t accessible.

With the Profile selection script completed, it’s time to assign the proper elements in the Inspector. Make sure ProfileSelection is attached to your Charprefab. Drag the BroadcasterSelectionPanel into the self-titled variable slot. Create a gold material called Gold by right-clicking in the Assets pane > Create > Material, and drag it into the Broadcaster Material slot. For the Viking Mesh, select Charprefab > Viking > BaseHuman.

In your BroadcasterSelectionPanel buttons, one assigns broadcaster status, and the other assigns audience status. For each button, if the OnClick() list is empty, click the plus sign (+) to create a function slot. Drag the Charprefab object into the slot. Click the dropdown titled “No Function > ProfileSelection > ButtonSetBroadcastState. For the Broadcaster button, make sure the check box is selected and the Audience check box is not selected.

Note: This toggle check box appears because our button function has a Boolean parameter, allowing you to assign the Boolean state from the Hierarchy UI.

Agora Profile Script

Create a script called AgoraProfile.cs and attach it to the Charprefab, just as you did for ProfileSelection.cs.

First, set up the variables necessary to drive the script, and inherit from Photon.PunBehaviour:

In the start method, you initialize the callbacks and necessary variables:

Next, create the join channel method:

Next, you will create the standard Agora callbacks. In this example, they are used for debugging. But it’s important to see how they work.

Note: I’ve tucked them in a #region so you can collapse the callbacks out of the way. I find regions very helpful. Feel free to use them or not!

Networked Speech Bubble

First, create two functions for toggling the speech bubble on and off:

Next, create the callback that fires during volume changes:

This checks if the local player is talking: If they are, place a networked speech bubble above their head. You want the speech bubble to appear as soon as the player starts talking, so no smoothing there.

This next function is designed to provide a slight delay before hiding the speech bubble, in case the broadcaster pauses, takes a breath, and so on.

The logic of the code looks like this:

  • When the player stops talking, activate the talkBubbleDisableTimer.
  • If the player starts talking again, reset the talkBubbleDisableTimer to its maximum value.
  • If the player stops talking until the timer hits 0, disable the speech bubble and exit the coroutine.

Note: The coroutine is “shielded” by multiple calls by checking if isSmoothingTalkBubble is true. If a coroutine is already running, all attempts to disable the talk bubble will be negated until the original coroutine is stopped.

Note: I’ve set the talkBubbleBuffer (starting timer amount) to .75f, which is the average human reaction time (.75 seconds). It’s a good default value to keep in mind for quality of life design tips like this, as anything significantly less might make the speech bubbles too jittery, and anything significantly more could make the speech bubble hang around too long. Play around with what you think is best!

One Last Thing. Before you conclude, you need to properly clean up the engine.

Include the callback:

In the Photon demo is a button that allows you to leave the lobby. This is really helpful for debugging to check different channels or player states, but it can cause errors if the Agora engine is running.

Conclusion

If you would like to contribute to this project and the broader Agora community as a whole, feel free to submit a pull request on GitHub and add your changes to the project!

If you need support, feel free to reach us on Slack here!