Interactivity with Player SDK

The Interactivity with Player SDKs can be implemented to provide a dynamic feature-rich user experience for iOS and Android.

Using Interactivity With The Brightcove Player SDK for iOS

The Brightcove Player SDK currently supports Interactivity in a limited capacity.

Supported Annotations:

Image Overlay
- Support for Command and Command Data.
- Support for “Jump to video time”.
- Support for opening URL. (Mobile Only).
Text Overlay
- White theme.
- Black “Mullins Special” theme.
- Support for Command and Command Data.
- Support for “Jump to video time”.
- Support for opening URL. (Mobile Only).
Time Triggered Action
- Support for Command and Command Data (Mobile and TV).
- Support for “Jump to video time” (Mobile and TV).
Transparent Overlay
- Support for Command and Command Data.
- Support for “Jump to video time” (Mobile and TV).
- Support for opening URL (Mobile Only).
Fade-in/fade-out transitions
Preconditions for dynamic showing and hiding of annotations
- Click X to show Y.
- Show Y on pause of the video.

Requirements

Brightcove Native Player SDK v6.12.6+
Brightcove Account with Dynamic Delivery.
Brightcove Account with Interactivity Enabled.

Limitations

Only the "Fade" transition is supported for Text and Image overlays.
Text, Transparent and Image overlays can be configured to be interactive on iOS, but this behavior is not yet supported on tvOS.
The "Mullins Special" and "White" themes are supported for Text overlay annotations.

Quick Start

Here is an example of adding Interactivity to your project:

self.interactivityHandler = [[BCOVInteractivityHandler alloc] initWithAccountId:kViewControllerAccountID projectId:kViewControllerProjectId containerView:self.playerView.interactivityOverlayView playbackController:self.playbackController];
self.interactivityHandler.delegate = self;

if let interactivityOverlayView = playerView.interactivityOverlayView, let playbackController = playbackController {
    interactivityHandler = BCOVInteractivityHandler(accountId: playbackConfig.accountID, projectId: playbackConfig.interactivityProjectID, containerView: interactivityOverlayView,  playbackController: playbackController)
    interactivityHandler?.delegate = self
}

Customization

The following delegate methods are available for customizing annotations:

- (CGFloat)animationTimeForTransition:(BCOVInteractivityAnnotationTransition)transition;
- (UIFont *)fontForTextAnnotation:(BCOVInteractivityAnnotation *)annotation;
- (UIColor *)backgroundColorForTextAnnotation:(BCOVInteractivityAnnotation *)annotation;
- (UIColor *)textColorForTextAnnotation:(BCOVInteractivityAnnotation *)annotation;
- (void)annotationWasTapped:(BCOVInteractivityAnnotation *)annotation; // iOS Only
- (void)annotationWasTriggered:(BCOVInteractivityAnnotation *)annotation;

Behavior

On iOS, if an annotation is set up to "Jump to video time" or launch a URL, it will do so without any additional work on your end. When the annotation is tapped, the `annotationWasTapped:` is called.

A Time-Triggered annotation will be triggered at the specified time, and the `annotationWasTriggered:` delegate method will be called. This delegate method can be used if you want additional behavior other than "Jump to video time".

You may also configure additional behavior on iOS by giving the annotation a Command value in the Interactivity editor (Edit > More > Link > Advanced). You may also supply additional values for the Command by adding key/value pairs to the Command Data.

Here's an example of displaying a Chapters menu using an `UIAlertController`:

- (void)annotationWasTapped:(BCOVInteractivityAnnotation *)annotation
{
    NSString *command = annotation.command;
    NSDictionary *commandData = annotation.commandData;

    if ([command isEqualToString:@"my-command"])
    {
        NSArray *sortedKeys = [commandData keysSortedByValueUsingComparator:^NSComparisonResult(id  _Nonnull obj1, id  _Nonnull obj2) {
            return [obj1 compare:obj2];
        }];

        NSMutableArray *sortedChapters = @[].mutableCopy;
        for (NSString *key in sortedKeys)
        {
            NSNumber *time = commandData[key];
            NSDictionary *chapter = @{
                @"title": key,
                @"time": time
            };
            [sortedChapters addObject:chapter];
        }

        UIAlertController *alert = [UIAlertController alertControllerWithTitle:@"Interactivity" message:@"Chapter Selection" preferredStyle:UIAlertControllerStyleActionSheet];

        for (NSDictionary *chapter in sortedChapters)
        {
            NSNumber *timeNum = chapter[@"time"];
            NSString *title = chapter[@"title"];
            CMTime time = CMTimeMake(timeNum.longLongValue, 1);
            [alert addAction:[UIAlertAction actionWithTitle:title style:UIAlertActionStyleDefault handler:^(UIAlertAction * _Nonnull action) {
                [self.playbackController seekToTime:time completionHandler:nil];
            }]];
        }

        [alert addAction:[UIAlertAction actionWithTitle:@"Cancel" style:UIAlertActionStyleCancel handler:nil]];

        alert.popoverPresentationController.sourceView = annotation.annotationView;

        [self presentViewController:alert animated:YES completion:nil];
    }
}

func annotationWasTapped(_ annotation: BCOVInteractivityAnnotation) {
    guard let command = annotation.command, let commandData = annotation.commandData else {
        return
    }

    if command == "my-command" {
        let sortedKeys = (commandData as NSDictionary).keysSortedByValue { (obj1, obj2) -> ComparisonResult in
            return (obj1 as! String).compare(obj2 as! String)
        }

        var sortedChapters: [NSDictionary] = []
        for key in sortedKeys {
            guard let key = key as? String, let timeStr = commandData[key] as? String, let time = NumberFormatter().number(from: timeStr) else {
                continue
            }
            let chapter: [String: Any] = [
                "title": key,
                "time": time
            ]
            sortedChapters.append(chapter as NSDictionary)
        }

        let alert = UIAlertController(title: "Interactivity", message: "Chapter Selection", preferredStyle: .actionSheet)

        for chapter in sortedChapters {
            if let timeNum = chapter["time"] as? NSNumber, let title = chapter["title"] as? String {
                let time = CMTimeMake(value: timeNum.int64Value, timescale: 1)
                alert.addAction(UIAlertAction(title: title, style: .default) { _ in
                    self.playbackController.seek(to: time, completionHandler: nil)
                })
            }
        }

        alert.addAction(UIAlertAction(title: "Cancel", style: .cancel, handler: nil))

        alert.popoverPresentationController?.sourceView = annotation.annotationView

        self.present(alert, animated: true, completion:nil)
    }
}

Preconditions

You can set up preconditions for your annotations to show and hide them on demand. Here is an example of hiding an annotation when playback begins and showing it five seconds after pause:

- (void)playbackController:(id)controller playbackSession:(id)session didReceiveLifecycleEvent:(BCOVPlaybackSessionLifecycleEvent *)lifecycleEvent
{
    if ([lifecycleEvent.eventType isEqualToString:kBCOVPlaybackSessionLifecycleEventPause])
    {
        self.xRayTimer = [NSTimer scheduledTimerWithTimeInterval:5 repeats:NO block:^(NSTimer * _Nonnull timer) {
            [self.interactivityHandler enablePrecondition:@"xRay"];
        }];
    }

    if ([lifecycleEvent.eventType isEqualToString:kBCOVPlaybackSessionLifecycleEventPlay])
    {
        [self.xRayTimer invalidate];
        [self.interactivityHandler disablePrecondition:@"xRay"];
    }
}

Using Interactivity With The Brightcove Player SDK for Android

The Android SDK's interactivity component allows annotations to be displayed on the player. These annotations must be previously set and configured for a video through Brightcove Studio. See the interactivity documentation to learn the basics of Interactivity and how to set annotations in Brightcove Studio.

Once the interactivity component is enabled in the Android SDK, the SDK will retrieve and display the annotations on the player view.

Annotations Supported

Image Overlay
- Command and Command Data.
- Jump to video time.
- Open URL.
Text Overlay
- White and Black “Mullins Special” theme.
- Command and Command Data.
- Jump to video time.
- Open URL.
Time-Triggered Action
- Command and Command Data.
- Jump to video time.
Transparent Overlay
- Command and Command Data.
- Jump to video time.
- Open URL.
Fade-in/fade-out transitions
Preconditions for dynamic showing and hiding of annotations

Setting Up Interactivity

The Android SDK allows interacting with interactivity’s functionalities through the InteractivityComponent class, which provides capabilities such as retrieving annotations from the backend, setting listeners for annotation events, enabling and disabling preconditions, etc.

Create an instance of InteractivityComponent

This component will help retrieve the annotations related to the video you want to play.
```
val interactivityComponent = InteractivityComponent(baseVideoView)
```

Retrieve Annotations

Retrieve the annotations for your video by passing the account ID and video ID. Use the getAnnotations() method to fetch this data.

interactivityComponent.getAnnotations(account = accountId, videoId = videoId, object : AnnotationsCallback<List<Annotation>> {
    override fun onSuccess(result: List<Annotation>) {
        Log.i("AnnotationsCallback", result.toString())
    }

    override fun onError(error: Throwable?) {
        error?.message?.let { Log.i("AnnotationsCallback", it) }
    }
})

Note: There is no need to call setAnnotations() method as the InteractivityComponent class automatically keeps the retrieved list.

Set Up Annotation Listener

To handle annotation events, set up an AnnotationListener. This listener provides information about various annotation-related events, such as when an annotation is displayed, removed, or interacted with.

//Annotations Listener
private val annotationsListener = object : AnnotationsListener {
    override fun onAnnotationDisplayed(annotation: Annotation) {
        Log.i(TAG, "onAnnotationDisplayed $annotation")
    }
    
    override fun onAnnotationRemoved(annotation: Annotation) {
        Log.i(TAG, "onAnnotationRemoved $annotation")
    }
    
    override fun onAnnotationTapped(annotation: Annotation) {
        Log.i(TAG, "onAnnotationTapped $annotation")
    }
    override fun onJumpToVideoTime(annotation: Annotation) {
        Log.i(TAG, "onJumpToVideoTime $annotation")
    }
    
    override fun onOpenURL(annotation: Annotation) {
        Log.i(TAG, "onOpenURL $annotation")
    }
}

interactivityComponent.setAnnotationsListener(annotationsListener)

Managing Preconditions

Preconditions can be used to dynamically show and hide annotations based on certain conditions. For example, if a precondition named "blue" is set in the Studio, you can enable it in the app as follows:
```
interactivityComponent.enablePrecondition("blue")
```
To disable the precondition:
```
interactivityComponent.disablePrecondition("blue")
```
Access Command and Command Data

In order to get the data for command and commandData in studio, you can access them through the annotation list returned on the onSuccess method like this:
```
override fun onSuccess(result: List) {
            val command: String? = result.find { it.id == "annotationId" }?.command
            val commandData: Map<String, String>? = result.find { it.id == "annotationId" }?.commandData
        }
```
In this example, we get the command and commandData values from the annotation list when the annotationId is equal to the one we are looking for.

Additionally, you can access command and commandData properties once an action occurs in an annotation (e.g., Displayed, hide, tapped, jumpedInTime, openURL) through the annotationsListener.
```
override fun onAnnotationTapped(annotation: Annotation) {
    val command: String = annotation.command
    val commandData: Map = annotation.commandData
}
```

Captions and Annotations

Annotations can be placed at the same place as captions, but captions are always in front of annotations. Even if annotations and captions are at the same position, annotations can be tapped to trigger the actions set in Studio.