The Google Assistant and Media Apps

The Google Assistant lets you use voice commands to control many devices, like Google Home, your phone, and more. It has a built-in capability to understand media commands ("play something by Beyonce") and supports media controls (like pause, skip, fast forward, thumbs up).

The Assistant communicates with Android media apps using a media session. It can use intents or services to launch your app and start playback. For the best results, your app should implement all the features described on this page.

This guide explains how to create a media app so that the Assistant can help a user control their media. To learn how Android apps use the Assist API to improve the assistant user experience, see Optimizing Content for the Assistant.

Use a media session

Every audio and video app must implement a media session so that the Assistant can operate the transport controls once playback has started.

Your app's media session must declare the actions it supports, and implement the corresponding media session callbacks. Declare your supported actions in setActions().

The Universal Music Player sample project is a good example of how to set up a media session.

Playback actions

In order to start playback from a service, a media session must have these these PLAY actions and their callbacks:

Your session should also implement these PREPARE actions and their callbacks:

By implementing the preparation APIs, the playback latency after a voice command can be reduced. Media apps that want to improve playback latency can use the extra time to start caching content and preparing media playback.

If onPrepare(), onPlay(), onPrepareFromSearch(), or onPlayFromSearch() are called without a search query, your media app should play the "current" media. If there is no current media, the app should try to play something.

Note that while the Assistant only uses the actions listed in this section, the best practice is to implement all preparation and playback APIs to ensure compatibility with other applications.

Transport controls

After your app's media session is active, the Assistant can issue voice commands to control playback and update media metadata. In order for this to work, your code should enable the following actions and implement the corresponding callbacks:

Please note:

• For seek commands to work, the PlaybackState needs to be up-to-date with the state, position, playback speed, and update time. The app must call setPlaybackState() when the state changes.
• The media app must also keep the media session metadata up-to-date. This supports questions such as "what song is playing?" The app must call setMetadata() when the applicable fields (such as track title, artist, and name) change.
• MediaSession.setRatingType() must be set to indicate the type of rating the app supports, and the app must implement onSetRating(). If the app does not support rating, it should set the rating type to RATING_NONE.

Errors

The Assistant handles errors from a media session when they occur, and reports them to users. Be sure that your media session updates the transport state and error code in its PlaybackState correctly, as described in Working with a media session. The Assistant recognizes all the error codes returned by getErrorCode().

Playback with an intent

The Assistant can launch an audio or video app and start playback by sending an intent with a deep link.

The intent and its deep link can come from different sources:

• When the Assistant is starting a mobile app, it can use Google search to retrieve marked up content that supplies a watch action with a link.
• When Assistant is starting a TV app, your app should include a TV Search Provider to expose URIs for media content. The Assistant sends a query to the content provider which should return an intent containing a URI for the deep link and an optional action. If the query returns an action in the intent, the Assistant sends that action and the URI back to the your app. If the provider did not specify an action, Assistant will add ACTION_VIEW to the Intent.

The Assistant adds the extra EXTRA_START_PLAYBACK with value true to the intent it sends to your app. Your app should start playback when it receives an intent with EXTRA_START_PLAYBACK.

Handling intents while active

Users can ask the Assistant to play something while your app is still playing content from a previous request. This means your app can receive new intents to start playback while its playback activity is already launched and active.

The activities that support intents with deep links should override onNewIntent() to handle new requests.

When starting playback, the Assistant might add additional flags to the intent it sends to your app. In particular, it may add FLAG_ACTIVITY_CLEAR_TOP or FLAG_ACTIVITY_NEW_TASK or both. Although your code does not need to handle these flags, the Android system responds to them. This might affect the behavior of your app when a second playback request with a new URI arrives while the previous URI is still playing. It's a good idea to test how your app responds in this case. You can use the adb command line tool to simulate the situation (the constant 0x14000000 is the boolean bitwise OR of the two flags):

adb shell 'am start -a android.intent.action.VIEW --ez android.intent.extra.START_PLAYBACK true -d <first_uri>' -f 0x14000000
adb shell 'am start -a android.intent.action.VIEW --ez android.intent.extra.START_PLAYBACK true -d <second_uri>' -f 0x14000000

Playback from a service

If your app has a media browser service that permits connections from the Assistant, the Assistant can start the app by communicating with the service's media session. The media browser service should never launch an Activity. The Assistant will launch your Activity based on the PendingIntent you define with setSessionActivity().

Be sure to set the MediaSession.Token when you initialize the media browser service. Remember to set the supported playback actions at all times, including during initialization. The Assistant expects your media app to set the playback actions before the Assistant sends the first playback command.

To start from a service, the Assistant implements the media browser client APIs. It performs TransportControls calls that trigger PLAY action callbacks on your app's media session.

The following diagram shows the order of calls generated by the Assistant and the corresponding media session callbacks. (The prepare callbacks are sent only if your app supports them.) All calls are asynchronous. The Assistant does not wait for any response from your app.

When a user issues a voice command to play, the Assistant responds with a short announcement. As soon as the announcement is complete the Assistant issues a PLAY action. It does not wait for any specific playback state.

If your app supports the ACTION_PREPARE_* actions, the Assistant calls the PREPARE action before starting the announcement.

Connecting to a MediaBrowserService

In order to use a service to start your app, the Assistant must be able to connect to the app's MediaBrowserService and retrieve its MediaSession.Token. Connection requests are handled in the service's onGetRoot() method. There are two ways to handle requests:

• Accept all connection requests
• Accept connection requests from the Assistant app only

Accept all connection requests

You must return a BrowserRoot in order to allow the Assistant to send commands to your MediaSession. The easiest way is to allow all MediaBrowser apps to connect to your MediaBrowserService. You must return a non-null BrowserRoot. Here is the applicable code from the Universal Music Player:

@Override
public BrowserRoot onGetRoot(@NonNull String clientPackageName, int clientUid,
                             Bundle rootHints) {

    // To ensure you are not allowing any arbitrary app to browse your app's contents, you
    // need to check the origin:
    if (!mPackageValidator.isCallerAllowed(this, clientPackageName, clientUid)) {
        // If the request comes from an untrusted package, return an empty browser root.
        // If you return null, then the media browser will not be able to connect and
        // no further calls will be made to other media browsing methods.
        LogHelper.i(TAG, "OnGetRoot: Browsing NOT ALLOWED for unknown caller. "
                + "Returning empty browser root so all apps can use MediaController."
                + clientPackageName);
        return new MediaBrowserServiceCompat.BrowserRoot(MEDIA_ID_EMPTY_ROOT, null);
    }

    // Return browser roots for browsing...
}

Accept the Assistant app package and signature

You can explicitly allow the Assistant to connect to your media browser service by checking for its package name and signature. Your app will receive the package name in the onGetRoot method of your MediaBrowserService. You must return a BrowserRoot in order to allow the Assistant to send commands to your MediaSession. The Universal Music Player sample maintains a list of known package names and signatures. Below are the package names and signatures that are used by the Google Assistant.

<signing_certificate name="Google" release="false"
                     package="com.google.android.googlequicksearchbox">
    MIIEqDCCA5CgAwIBAgIJANWFuGx90071MA0GCSqGSIb3DQEBBAUAMIGUMQswCQYD
    VQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTEWMBQGA1UEBxMNTW91bnRhaW4g
    VmlldzEQMA4GA1UEChMHQW5kcm9pZDEQMA4GA1UECxMHQW5kcm9pZDEQMA4GA1UE
    AxMHQW5kcm9pZDEiMCAGCSqGSIb3DQEJARYTYW5kcm9pZEBhbmRyb2lkLmNvbTAe
    Fw0wODA0MTUyMzM2NTZaFw0zNTA5MDEyMzM2NTZaMIGUMQswCQYDVQQGEwJVUzET
    MBEGA1UECBMKQ2FsaWZvcm5pYTEWMBQGA1UEBxMNTW91bnRhaW4gVmlldzEQMA4G
    A1UEChMHQW5kcm9pZDEQMA4GA1UECxMHQW5kcm9pZDEQMA4GA1UEAxMHQW5kcm9p
    ZDEiMCAGCSqGSIb3DQEJARYTYW5kcm9pZEBhbmRyb2lkLmNvbTCCASAwDQYJKoZI
    hvcNAQEBBQADggENADCCAQgCggEBANbOLggKv+IxTdGNs8/TGFy0PTP6DHThvbbR
    24kT9ixcOd9W+EaBPWW+wPPKQmsHxajtWjmQwWfna8mZuSeJS48LIgAZlKkpFeVy
    xW0qMBujb8X8ETrWy550NaFtI6t9+u7hZeTfHwqNvacKhp1RbE6dBRGWynwMVX8X
    W8N1+UjFaq6GCJukT4qmpN2afb8sCjUigq0GuMwYXrFVee74bQgLHWGJwPmvmLHC
    69EH6kWr22ijx4OKXlSIx2xT1AsSHee70w5iDBiK4aph27yH3TxkXy9V89TDdexA
    cKk/cVHYNnDBapcavl7y0RiQ4biu8ymM8Ga/nmzhRKya6G0cGw8CAQOjgfwwgfkw
    HQYDVR0OBBYEFI0cxb6VTEM8YYY6FbBMvAPyT+CyMIHJBgNVHSMEgcEwgb6AFI0c
    xb6VTEM8YYY6FbBMvAPyT+CyoYGapIGXMIGUMQswCQYDVQQGEwJVUzETMBEGA1UE
    CBMKQ2FsaWZvcm5pYTEWMBQGA1UEBxMNTW91bnRhaW4gVmlldzEQMA4GA1UEChMH
    QW5kcm9pZDEQMA4GA1UECxMHQW5kcm9pZDEQMA4GA1UEAxMHQW5kcm9pZDEiMCAG
    CSqGSIb3DQEJARYTYW5kcm9pZEBhbmRyb2lkLmNvbYIJANWFuGx90071MAwGA1Ud
    EwQFMAMBAf8wDQYJKoZIhvcNAQEEBQADggEBABnTDPEF+3iSP0wNfdIjIz1AlnrP
    zgAIHVvXxunW7SBrDhEglQZBbKJEk5kT0mtKoOD1JMrSu1xuTKEBahWRbqHsXcla
    XjoBADb0kkjVEJu/Lh5hgYZnOjvlba8Ld7HCKePCVePoTJBdI4fvugnL8TsgK05a
    IskyY0hKI9L8KfqfGTl1lzOv2KoWD0KWwtAWPoGChZxmQ+nBli+gwYMzM1vAkP+a
    ayLe0a1EQimlOalO762r0GXO0ks+UeXde2Z4e+8S/pf7pITEI/tP+MxJTALw9QUW
    Ev9lKTk+jkbqxbsh8nfBUapfKqYn0eidpwq2AzVp3juYl7//fKnaPhJD9gs=
</signing_certificate>

<signing_certificate name="Google" release="true"
                     package="com.google.android.googlequicksearchbox">
    MIIEQzCCAyugAwIBAgIJAMLgh0ZkSjCNMA0GCSqGSIb3DQEBBAUAMHQxCzAJBgNV
    BAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1Nb3VudGFpbiBW
    aWV3MRQwEgYDVQQKEwtHb29nbGUgSW5jLjEQMA4GA1UECxMHQW5kcm9pZDEQMA4G
    A1UEAxMHQW5kcm9pZDAeFw0wODA4MjEyMzEzMzRaFw0zNjAxMDcyMzEzMzRaMHQx
    CzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1Nb3Vu
    dGFpbiBWaWV3MRQwEgYDVQQKEwtHb29nbGUgSW5jLjEQMA4GA1UECxMHQW5kcm9p
    ZDEQMA4GA1UEAxMHQW5kcm9pZDCCASAwDQYJKoZIhvcNAQEBBQADggENADCCAQgC
    ggEBAKtWLgDYO6IIrgqWbxJOKdoR8qtW0I9Y4sypEwPpt1TTcvZApxsdyxMJZ2JO
    Rland2qSGT2y5b+3JKkedxiLDmpHpDsz2WCbdxgxRczfey5YZnTJ4VZbH0xqWVW/
    8lGmPav5xVwnIiJS6HXk+BVKZF+JcWjAsb/GEuq/eFdpuzSqeYTcfi6idkyugwfY
    wXFU1+5fZKUaRKYCwkkFQVfcAs1fXA5V+++FGfvjJ/CxURaSxaBvGdGDhfXE28LW
    uT9ozCl5xw4Yq5OGazvV24mZVSoOO0yZ31j7kYvtwYK6NeADwbSxDdJEqO4k//0z
    OHKrUiGYXtqw/A0LFFtqoZKFjnkCAQOjgdkwgdYwHQYDVR0OBBYEFMd9jMIhF1Yl
    mn/Tgt9r45jk14alMIGmBgNVHSMEgZ4wgZuAFMd9jMIhF1Ylmn/Tgt9r45jk14al
    oXikdjB0MQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTEWMBQGA1UE
    BxMNTW91bnRhaW4gVmlldzEUMBIGA1UEChMLR29vZ2xlIEluYy4xEDAOBgNVBAsT
    B0FuZHJvaWQxEDAOBgNVBAMTB0FuZHJvaWSCCQDC4IdGZEowjTAMBgNVHRMEBTAD
    AQH/MA0GCSqGSIb3DQEBBAUAA4IBAQBt0lLO74UwLDYKqs6Tm8/yzKkEu116FmH4
    rkaymUIE0P9KaMftGlMexFlaYjzmB2OxZyl6euNXEsQH8gjwyxCUKRJNexBiGcCE
    yj6z+a1fuHHvkiaai+KL8W1EyNmgjmyy8AW7P+LLlkR+ho5zEHatRbM/YAnqGcFh
    5iZBqpknHf1SKMXFh4dd239FJ1jWYfbMDMy3NS5CTMQ2XFI1MvcyUTdZPErjQfTb
    Qe3aDQsQcafEQPD+nqActifKZ0Np0IS9L9kR/wbNvyz6ENwPiTrjV2KRkEjH78ZM
    cUQXg0L3BYHJ3lc69Vs5Ddf9uUGGMYldX3WfMBEmh/9iFBDAaTCK
</signing_certificate>