firebase-ai-logic-basics

# Flutter Setup for Firebase AI Logic

This guide covers how to integrate Firebase AI Logic (Gemini API) into your
Flutter applications.

> [!IMPORTANT] **Foundational Workflows & CLI-First Approach:**
>
> 1. **Review Foundation:** Before implementing platform-specific code, ALWAYS
>    review the foundational `firebase-basics` skill to ensure familiarity with
>    core workflows.
> 1. **Backend Provisioning via CLI:** Use the Firebase CLI for backend setup.
>    Running `npx firebase-tools init ailogic` is MANDATORY to provision the
>    service. `flutterfire configure` does NOT enable the AI service and will
>    result in `PERMISSION_DENIED` if skipped.
> 1. **Client Configuration:** Use `flutterfire configure` strictly for
>    generating `firebase_options.dart`. Avoid manual Console configuration.

> [!NOTE] `firebase_vertexai` has been replaced by `firebase_ai`. Always use
> `firebase_ai` for new projects.

## Installation

Add the necessary Firebase dependencies to your `pubspec.yaml`:

```yaml
dependencies:
  flutter:
    sdk: flutter
  firebase_core: ^4.0.0
  firebase_auth: ^6.0.0
  firebase_ai: ^3.0.0
```

Run `flutter pub get` to install the packages.

## Initialization

Initialize Firebase and sign in (anonymously or via authenticated user) before
using AI Logic.

```dart
import 'package:firebase_core/firebase_core.dart';
import 'package:firebase_auth/firebase_auth.dart';
import 'package:firebase_ai/firebase_ai.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp();
  await FirebaseAuth.instance.signInAnonymously();
  runApp(const MyApp());
}
```

## Usage

Use `FirebaseAI.googleAI` for the **Gemini Developer API**.

> [!IMPORTANT] **Model Selection:** Refer to
> [Firebase AI Logic Models](https://firebase.google.com/docs/ai-logic/models.md.txt)
> to find the latest supported model. Do not use old models like
> `gemini-2.0-pro` or `gemini-2.0-flash`.

> [!IMPORTANT] **Choose the Right API Provider:** Always use
> `FirebaseAI.googleAI` (Gemini Developer API) as the default for prototyping
> and standard use. Avoid using the Agent Platform Gemini API (formerly branded
> Vertex AI) unless the app and business use case specifically require
> enterprise-grade scalability or data residency requirements. Note that the
> Gemini Developer API *usually does not* require the Firebase project to be on
> the pay-as-you-go Blaze pricing plan; however, the Agent Platform Gemini API
> does require the Blaze plan.

### Text Generation

```dart
import 'package:firebase_ai/firebase_ai.dart';
import 'package:firebase_auth/firebase_auth.dart';

Future<String> generateText(String prompt) async {
  final googleAI = FirebaseAI.googleAI(auth: FirebaseAuth.instance);
  
  // [AGENT] Replace '<latest_supported_model>' with the latest model from https://firebase.google.com/docs/ai-logic/models.md.txt
  final model = googleAI.generativeModel(model: '<latest_supported_model>');

  final response = await model.generateContent([Content.text(prompt)]);
  return response.text ?? 'No response';
}
```

### Chat Session

```dart
final chat = model.startChat(history: [
  Content.text('Hello, I am a user.'),
  Content.model([TextPart('Hello! How can I help you today?')]),
]);

final response = await chat.sendMessage(Content.text('What is CBT?'));
```

# Firebase AI Logic iOS Setup Guide

## 1. Import and Initialize

Ensure you have installed the `FirebaseAILogic` SDK via Swift Package Manager.

```swift
import FirebaseAILogic

// To enable replay protection for generative or preview models, pass
// useLimitedUseAppCheckTokens: true with your backend:
let ai = FirebaseAI.firebaseAI(
    backend: .googleAI(),
    useLimitedUseAppCheckTokens: true
)

// [AGENT] Replace "<latest_supported_model>" with the latest model from https://firebase.google.com/docs/ai-logic/models.md.txt
let model = ai.generativeModel(modelName: "<latest_supported_model>")
```

### App Check Replay Protection

Generative and preview models enforce replay protection with 5-minute
limited-use App Check tokens. If you call a protected model without enabling
limited-use tokens, the request fails with:

```text
HTTP 403: "To access this model, you must enforce Firebase App Check"
```

To resolve this error, initialize `FirebaseAI` with
`useLimitedUseAppCheckTokens: true`:

```swift
let ai = FirebaseAI.firebaseAI(
    backend: .googleAI(),
    useLimitedUseAppCheckTokens: true
)
```

This ensures the SDK fetches a fresh, short-lived limited-use token for each
request rather than reusing a standard cached App Check token.

## 2. SwiftUI Integration (Best Practices)

Use the `@Observable` pattern to manage AI state and provide a smooth UX with
loading indicators and error handling.

> **⛔️ CRITICAL WARNING:** Do NOT initialize the model inline as a class
> property if there's any chance the view model is instantiated before
> `FirebaseApp.configure()` executes in the app root. To be safe, initialize the
> model lazily or pass it in from a point in the hierarchy where Firebase is
> guaranteed to be configured.

```swift
import SwiftUI
import FirebaseAILogic

@MainActor
@Observable
final class AIViewModel {
    // [AGENT] Replace with the latest model from https://firebase.google.com/docs/ai-logic/models.md.txt
    private lazy var model = FirebaseAI.firebaseAI(
        backend: .googleAI(),
        useLimitedUseAppCheckTokens: true
    ).generativeModel(modelName: "<latest_supported_model>")
    
    var responseText: String = ""
    var isFetching: Bool = false
    var errorMessage: String?
    
    func generate(prompt: String) async {
        isFetching = true
        errorMessage = nil
        defer { isFetching = false }
        
        do {
            let response = try await model.generateContent(prompt)
            self.responseText = response.text ?? "No response"
        } catch {
            self.errorMessage = error.localizedDescription
        }
    }
}

struct AIView: View {
    @State private var viewModel = AIViewModel()
    @State private var prompt = "Write a story about a magic backpack."
    
    var body: some View {
        VStack {
            TextField("Enter prompt", text: $prompt)
            
            Button("Generate") {
                Task { await viewModel.generate(prompt: prompt) }
            }
            .disabled(viewModel.isFetching)
            
            if viewModel.isFetching {
                ProgressView()
            } else if let error = viewModel.errorMessage {
                Text(error).foregroundStyle(.red)
            } else {
                ScrollView {
                    Text(viewModel.responseText)
                }
            }
        }
        .padding()
    }
}
```

## 3. Safety Settings

You can configure safety thresholds to prevent the model from generating harmful
content.

```swift
let safetySettings = [
  SafetySetting(harmCategory: .harassment, threshold: .blockLowAndAbove),
  SafetySetting(harmCategory: .hateSpeech, threshold: .blockMediumAndAbove)
]

let model = FirebaseAI.firebaseAI().generativeModel(
  modelName: "<latest_supported_model>", // [AGENT] Replace with the latest model from https://firebase.google.com/docs/ai-logic/models.md.txt
  safetySettings: safetySettings
)
```

# Advanced Features

### Chat Session (Multi-turn)

Chat sessions persist state across multiple interactions, which is essential for
ongoing conversations or when using tools like function calling.

```swift
let chat = model.startChat()

Task {
    do {
        let response1 = try await chat.sendMessage("Hello! I have two dogs in my house.")
        print(response1.text ?? "")

        let response2 = try await chat.sendMessage("How many paws are in my house?")
        print(response2.text ?? "")
    } catch {
        print("Error in chat: \(error)")
    }
}
```

### Function Calling (Tools)

Define functions that the model can request to execute to interact with external
systems. *Note: Advanced workflows like function calling generally require a
multi-turn Chat Session to handle the back-and-forth execution.*

```swift
let getStockPriceTool = Tool.functionDeclarations([
  FunctionDeclaration(
    name: "getStockPrice",
    description: "Get the current stock price for a given symbol.",
    parameters: [
      "symbol": .string(
        description: "The stock symbol, e.g. AAPL"
      )
    ]
  )
])

let model = FirebaseAI.firebaseAI().generativeModel(
  modelName: "<latest_supported_model>", // [AGENT] Replace with the latest model from https://firebase.google.com/docs/ai-logic/models.md.txt
  tools: [getStockPriceTool]
)

// In your task (using a chat session):
let chat = model.startChat()
let response = try await chat.sendMessage("What is the stock price of Apple?")
if let functionCall = response.functionCalls.first {
    // Handle the function call (e.g. call a local API and send the result back)
    print("Model requested function: \(functionCall.name) with args: \(functionCall.args)")
}
```

### App Check (Debug Token Persistence)

When running on simulators or during development, the Firebase iOS SDK generates
a new debug token UUID whenever `NSUserDefaults` is cleared (e.g., simulator
reset or fresh install). To avoid invalidating tokens registered in the Firebase
Console and prevent token churn, persist a stable token by setting the
`AppCheckDebugToken` environment variable:

> [!WARNING] **CRITICAL: Never Hardcode or Commit Debug Tokens** Debug tokens
> grant access to backend resources without device attestation. Never commit
> debug tokens to version control or hardcode token strings in source code.

- **In Xcode Scheme (Recommended):** Edit Scheme -> Run -> Arguments ->
  Environment Variables -> Add `AppCheckDebugToken = <YOUR_DEBUG_TOKEN>`. Keep
  user schemes (`xcuserdata/`) unshared and gitignored.
- **In Code (Safe Dynamic Loading Only):** If setting the environment variable
  in code before configuring `AppCheckDebugProviderFactory`, load the token
  dynamically from a gitignored local file or environment rather than hardcoding
  the token literal:

```swift
#if DEBUG
// ✅ SAFE: Load from gitignored local file or process environment
// Note: loadGitIgnoredDebugToken() is a placeholder for your custom helper (e.g., reading from a gitignored plist)
if let debugToken = loadGitIgnoredDebugToken() {
  setenv("AppCheckDebugToken", debugToken, 0)
}
let providerFactory = AppCheckDebugProviderFactory()
AppCheck.setAppCheckProviderFactory(providerFactory)
#endif

FirebaseApp.configure() // Configure Firebase AFTER setting the App Check provider factory
```

# Client-Side Text-to-Speech (TTS) Generation with Gemini

Firebase AI Logic enables client-side Text-to-Speech (TTS) generation directly
from your mobile and web applications without maintaining custom backend speech
services. Using specialized Gemini TTS models, apps can synthesize natural,
expressive audio with custom voice personas, multi-speaker dialogues, and
in-flight audio streaming.

______________________________________________________________________

## Supported Models

Always specify a dedicated Gemini TTS model when generating audio:

| Model ID                       | Description                                                  |
| :----------------------------- | :----------------------------------------------------------- |
| `gemini-3.8-flash-tts`         | High-quality, production-ready low-latency speech synthesis. |
| `gemini-3.1-flash-tts-preview` | Preview model for experimental voice synthesis capabilities. |

> [!WARNING] Always verify currently supported model availability in the
> [Firebase AI Logic Models documentation](https://firebase.google.com/docs/ai-logic/models.md.txt).

______________________________________________________________________

## Core Concepts & Configurations

### 1. Response Modality

To instruct Gemini to synthesize and return audio, configure
`responseModalities` in the generation configuration to include audio:

- **Swift**: `generationConfig = GenerationConfig(responseModalities: [.audio])`
- **Kotlin**:
  `generationConfig { responseModalities = listOf(ResponseModality.AUDIO) }`
- **Web (JS/TS)**: `generationConfig: { responseModalities: ["AUDIO"] }`

### 2. Single-Speaker `SpeechConfig`

Configure voice persona and optional language code (`languageCode`, e.g.
`"en-US"`, `"es-ES"`, `"ja-JP"`) using `SpeechConfig`. Voice names map to Google
prebuilt neural voices (for example: `Puck`, `Charon`, `Kore`, `Fenrir`,
`Aoede`):

- **Swift**:
  ```swift
  let speechConfig = SpeechConfig(
      voiceConfig: VoiceConfig(
          prebuiltVoiceConfig: PrebuiltVoiceConfig(voiceName: "Puck")
      ),
      languageCode: "en-US"
  )
  ```
- **Kotlin**:
  ```kotlin
  val speechConfig = speechConfig {
      voiceConfig = voiceConfig {
          prebuiltVoiceConfig = prebuiltVoiceConfig {
              voiceName = "Puck"
          }
      }
      languageCode = "en-US"
  }
  ```
- **Web**:
  ```javascript
  speechConfig: {
    voiceConfig: {
      prebuiltVoiceConfig: {
        voiceName: "Puck"
      }
    },
    languageCode: "en-US"
  }
  ```

### 3. Multi-Speaker `MultiSpeakerVoiceConfig` (2-Speaker Dialogues)

For scripts with two distinct characters or roles (e.g., host and guest,
narrator and character), configure `MultiSpeakerVoiceConfig` with
`speakerVoiceConfigs` linking speaker names to specific voices:

- **Swift**:
  ```swift
  let multiSpeakerConfig = SpeechConfig(
      multiSpeakerVoiceConfig: MultiSpeakerVoiceConfig(
          speakerVoiceConfigs: [
              SpeakerVoiceConfig(
                  speaker: "Host",
                  voiceConfig: VoiceConfig(
                      prebuiltVoiceConfig: PrebuiltVoiceConfig(voiceName: "Puck")
                  )
              ),
              SpeakerVoiceConfig(
                  speaker: "Guest",
                  voiceConfig: VoiceConfig(
                      prebuiltVoiceConfig: PrebuiltVoiceConfig(voiceName: "Aoede")
                  )
              )
          ]
      )
  )
  ```
- **Kotlin**:
  ```kotlin
  val multiSpeakerConfig = speechConfig {
      multiSpeakerVoiceConfig = multiSpeakerVoiceConfig {
          speakerVoiceConfigs = listOf(
              speakerVoiceConfig {
                  speaker = "Host"
                  voiceConfig = voiceConfig {
                      prebuiltVoiceConfig = prebuiltVoiceConfig { voiceName = "Puck" }
                  }
              },
              speakerVoiceConfig {
                  speaker = "Guest"
                  voiceConfig = voiceConfig {
                      prebuiltVoiceConfig = prebuiltVoiceConfig { voiceName = "Aoede" }
                  }
              }
          )
      }
  }
  ```
- **Web**:
  ```javascript
  speechConfig: {
    multiSpeakerVoiceConfig: {
      speakerVoiceConfigs: [
        {
          speaker: "Host",
          voiceConfig: { prebuiltVoiceConfig: { voiceName: "Puck" } }
        },
        {
          speaker: "Guest",
          voiceConfig: { prebuiltVoiceConfig: { voiceName: "Aoede" } }
        }
      ]
    }
  }
  ```

______________________________________________________________________

## Audio Directives and Emotional Tags

Gemini TTS models accept natural-language directives and inline tags in prompts
to control tone, cadence, pacing, and emotional expression.

### Directives

Include directives at the start of your prompt to set the overall tone and
scene:

- `[Audio Profile: Warm, conversational podcast co-host with energetic delivery]`
- `[Scene: Quiet bedtime story in a calm room]`
- `[Director's Note: Speak deliberately, pausing for reflection after major questions]`

### Emotional Tags & Delivery Cues

Embed delivery markers inline within dialogue or narration:

- `[whispers]` — Soft, confidential whisper.
- `[laughs]` / `[chuckles]` — Lighthearted laughter during delivery.
- `[sighs]` — Expressive exhale.
- `[slowly]` — Reduced tempo for emphasis or dramatic effect.
- `[excited]` — Higher energy and faster pacing.
- `[pause]` — Deliberate silence.

### Multi-Speaker Prompt Example

```text
[Scene: A lively coffee-shop tech discussion]
[Director's Note: Host is upbeat and curious; Guest is thoughtful and explanatory]

Host: Welcome back! [excited] Today we are diving into client-side audio.
Guest: [laughs] It's about time! No servers in between means virtually zero latency.
Host: [whispers] Tell us the secret... how does it stream?
Guest: [slowly] Pure linear PCM, chunk by chunk, directly to your device's audio engine.
```

______________________________________________________________________

## Audio Decoding and Playback

### Format Specifications

Gemini TTS audio format depends on whether the request is unary or streaming:

- **Unary Requests (`generateContent`)**: Returns complete **WAV (`audio/wav`)**
  audio with a standard 44-byte RIFF header included by default (24 kHz, mono,
  16-bit signed little-endian PCM). These bytes can be passed directly to
  standard media players (`AVAudioPlayer`, `MediaPlayer`, `<audio>`).

- **Streaming Requests (`generateContentStream`)**: Returns headerless raw
  **Linear PCM (`audio/l16; rate=24000; channels=1`)** chunks (24 kHz, mono,
  16-bit signed little-endian PCM) so chunks can be streamed or concatenated
  continuously without per-chunk container headers.

- **Audio Parameters**:

  - **Sample Rate**: 24,000 Hz (24 kHz)
  - **Channels**: 1 channel (mono)
  - **Bit Depth**: 16-bit linear PCM (little-endian signed integer)
  - **MIME Types**: `audio/l16` (streaming chunks) or `audio/wav` /
    `audio/x-wav` (unary)

### Playback Approaches

1. **Direct Low-Latency PCM Streaming**:
   - Stream raw PCM chunks directly into low-level audio renderers without
     waiting for the full response:
     - **iOS**: `AVAudioEngine` + `AVAudioPlayerNode` scheduling
       `AVAudioPCMBuffer`.
     - **Android**: `AudioTrack` configured with `ENCODING_PCM_16BIT` and
       `CHANNEL_OUT_MONO`.
     - **Web**: Web Audio API (`AudioContext`) scheduling PCM chunks into
       `AudioBufferSourceNode`.
1. **WAV Container Format**:
   - Standard media players (`AVAudioPlayer` on iOS, `MediaPlayer` on Android,
     `<audio>` on Web) expect a container header (RIFF/WAV).
   - For unary responses, the RIFF header is already present. For assembled
     streaming PCM chunks, prepend a standard **44-byte RIFF header** to make
     the complete audio immediately playable or saveable as `.wav`.

#### Standard 44-Byte WAV (RIFF) Header Construction

```
Offset  Size  Field              Value
0       4     ChunkID            "RIFF"
4       4     ChunkSize          36 + Subchunk2Size (file size - 8)
8       4     Format             "WAVE"
12      4     Subchunk1ID        "fmt "
16      4     Subchunk1Size      16 (for PCM)
20      2     AudioFormat        1 (PCM linear)
22      2     NumChannels        1 (Mono)
24      4     SampleRate         24000
28      4     ByteRate           48000 (SampleRate * NumChannels * BitsPerSample / 8)
32      2     BlockAlign         2 (NumChannels * BitsPerSample / 8)
34      2     BitsPerSample      16
36      4     Subchunk2ID        "data"
40      4     Subchunk2Size      Number of PCM bytes
44+     N     Data               Raw PCM bytes
```

______________________________________________________________________

## Platform Code Snippets

### Swift (iOS)

#### 1. Streaming Playback with `AVAudioEngine`

```swift
import AVFoundation
import FirebaseAILogic

@MainActor
final class SpeechStreamingManager {
    private let audioEngine = AVAudioEngine()
    private let playerNode = AVAudioPlayerNode()
    
    // 24 kHz, 16-bit Mono PCM
    private let audioFormat = AVAudioFormat(
        commonFormat: .pcmFormatInt16,
        sampleRate: 24000,
        channels: 1,
        interleaved: false
    )!

    init() {
        audioEngine.attach(playerNode)
        audioEngine.connect(playerNode, to: audioEngine.mainMixerNode, format: audioFormat)
    }

    func streamSpeech(prompt: String) async throws {
        let ai = FirebaseAI.firebaseAI()
        
        let speechConfig = SpeechConfig(
            voiceConfig: VoiceConfig(
                prebuiltVoiceConfig: PrebuiltVoiceConfig(voiceName: "Puck")
            )
        )
        
        let genConfig = GenerationConfig(
            responseModalities: [.audio],
            speechConfig: speechConfig
        )
        
        let model = ai.generativeModel(
            modelName: "gemini-3.8-flash-tts",
            generationConfig: genConfig
        )
        
        if !audioEngine.isRunning {
            try audioEngine.start()
        }
        playerNode.play()

        let stream = model.generateContentStream(prompt)
        for try await chunk in stream {
            for part in chunk.candidates.first?.content.parts ?? [] {
                if let inlineData = part as? InlineDataPart,
                   inlineData.mimeType.contains("audio/pcm") || inlineData.mimeType.contains("audio/l16") {
                    schedulePCMBuffer(data: inlineData.data)
                }
            }
        }
    }

    private func schedulePCMBuffer(data: Data) {
        let frameCount = UInt32(data.count) / audioFormat.streamDescription.pointee.mBytesPerFrame
        guard frameCount > 0,
              let buffer = AVAudioPCMBuffer(pcmFormat: audioFormat, frameCapacity: frameCount) else {
            return
        }
        buffer.frameLength = frameCount
        
        data.withUnsafeBytes { rawBufferPointer in
            guard let src = rawBufferPointer.baseAddress else { return }
            if let dest = buffer.int16ChannelData?[0] {
                UnsafeMutableRawPointer(dest).copyMemory(from: src, byteCount: data.count)
            }
        }
        
        playerNode.scheduleBuffer(buffer)
    }

    func stop() {
        playerNode.stop()
        audioEngine.stop()
    }
}
```

#### 2. WAV Converter for `AVAudioPlayer`

```swift
func addWavHeader(to pcmData: Data, sampleRate: Int = 24000, channels: Int = 1, bitDepth: Int = 16) -> Data {
    var header = Data()
    let byteRate = sampleRate * channels * bitDepth / 8
    let blockAlign = channels * bitDepth / 8
    let totalDataLen = Int32(pcmData.count)
    let totalLength = totalDataLen + 36

    header.append(contentsOf: "RIFF".utf8)
    header.append(Data(from: totalLength.littleEndian))
    header.append(contentsOf: "WAVE".utf8)
    header.append(contentsOf: "fmt ".utf8)
    header.append(Data(from: Int32(16).littleEndian))       // Subchunk1Size for PCM
    header.append(Data(from: Int16(1).littleEndian))        // AudioFormat 1 = PCM
    header.append(Data(from: Int16(channels).littleEndian))
    header.append(Data(from: Int32(sampleRate).littleEndian))
    header.append(Data(from: Int32(byteRate).littleEndian))
    header.append(Data(from: Int16(blockAlign).littleEndian))
    header.append(Data(from: Int16(bitDepth).littleEndian))
    header.append(contentsOf: "data".utf8)
    header.append(Data(from: totalDataLen.littleEndian))

    return header + pcmData
}

private extension Data {
    init<T>(from value: T) {
        var val = value
        self = Swift.withUnsafeBytes(of: &val) { Data($0) }
    }
}
```

______________________________________________________________________

### Kotlin (Android)

#### Streaming Playback with `AudioTrack`

```kotlin
import android.media.AudioAttributes
import android.media.AudioFormat
import android.media.AudioTrack
import com.google.firebase.ai.FirebaseAI
import com.google.firebase.ai.type.ResponseModality
import com.google.firebase.ai.type.generationConfig
import com.google.firebase.ai.type.prebuiltVoiceConfig
import com.google.firebase.ai.type.speechConfig
import com.google.firebase.ai.type.voiceConfig
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext

class SpeechStreamingManager {
    private val sampleRate = 24000
    private val minBufferSize = AudioTrack.getMinBufferSize(
        sampleRate,
        AudioFormat.CHANNEL_OUT_MONO,
        AudioFormat.ENCODING_PCM_16BIT
    )

    private val audioTrack = AudioTrack.Builder()
        .setAudioAttributes(
            AudioAttributes.Builder()
                .setContentType(AudioAttributes.CONTENT_TYPE_SPEECH)
                .setUsage(AudioAttributes.USAGE_MEDIA)
                .build()
        )
        .setAudioFormat(
            AudioFormat.Builder()
                .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
                .setSampleRate(sampleRate)
                .setChannelMask(AudioFormat.CHANNEL_OUT_MONO)
                .build()
        )
        .setBufferSizeInBytes(minBufferSize * 2)
        .setTransferMode(AudioTrack.MODE_STREAM)
        .build()

    suspend fun streamSpeech(prompt: String) = withContext(Dispatchers.IO) {
        val config = generationConfig {
            responseModalities = listOf(ResponseModality.AUDIO)
            speechConfig = speechConfig {
                voiceConfig = voiceConfig {
                    prebuiltVoiceConfig = prebuiltVoiceConfig {
                        voiceName = "Puck"
                    }
                }
            }
        }

        val generativeModel = FirebaseAI.getInstance().getGenerativeModel(
            modelName = "gemini-3.8-flash-tts",
            generationConfig = config
        )

        audioTrack.play()
        try {
            val responseStream = generativeModel.generateContentStream(prompt)
            responseStream.collect { chunk ->
                chunk.candidates.firstOrNull()?.content?.parts?.forEach { part ->
                    if (part is com.google.firebase.ai.type.InlineDataPart) {
                        val audioBytes = part.data
                        audioTrack.write(audioBytes, 0, audioBytes.size)
                    }
                }
            }
        } finally {
            audioTrack.stop()
        }
    }

    fun release() {
        audioTrack.release()
    }
}
```

#### 2. Convert PCM to WAV for `MediaPlayer`

```kotlin
import android.content.Context
import android.media.MediaPlayer
import java.io.File
import java.io.FileOutputStream
import java.nio.ByteBuffer
import java.nio.ByteOrder

fun addWavHeader(
    pcmBytes: ByteArray,
    sampleRate: Int = 24000,
    channels: Short = 1,
    bitDepth: Short = 16
): ByteArray {
    val totalDataLen = pcmBytes.size
    val totalLength = totalDataLen + 36
    val byteRate = sampleRate * channels * bitDepth / 8
    val blockAlign = (channels * bitDepth / 8).toShort()

    val header = ByteBuffer.allocate(44).order(ByteOrder.LITTLE_ENDIAN).apply {
        put("RIFF".toByteArray())
        putInt(totalLength)
        put("WAVE".toByteArray())
        put("fmt ".toByteArray())
        putInt(16) // Subchunk1Size
        putShort(1) // AudioFormat 1 = PCM
        putShort(channels)
        putInt(sampleRate)
        putInt(byteRate)
        putShort(blockAlign)
        putShort(bitDepth)
        put("data".toByteArray())
        putInt(totalDataLen)
    }.array()

    return header + pcmBytes
}

fun playWavAudio(context: Context, wavBytes: ByteArray) {
    val tempFile = File.createTempFile("tts_", ".wav", context.cacheDir).apply {
        deleteOnExit()
        FileOutputStream(this).use { it.write(wavBytes) }
    }

    val mediaPlayer = MediaPlayer()
    try {
        mediaPlayer.setDataSource(tempFile.absolutePath)
        mediaPlayer.prepare()
        mediaPlayer.start()
        mediaPlayer.setOnCompletionListener {
            it.release()
            tempFile.delete()
        }
    } catch (e: Exception) {
        mediaPlayer.release()
        tempFile.delete()
        throw e
    }
}
```

______________________________________________________________________

### Web (JavaScript/TypeScript)

#### 1. Streaming Playback with Web Audio API

```typescript
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend } from "firebase/ai";

const firebaseConfig = {
  // your firebase configuration
};

const app = initializeApp(firebaseConfig);
const ai = getAI(app, { backend: new GoogleAIBackend() });

const model = getGenerativeModel(ai, {
  model: "gemini-3.8-flash-tts",
  generationConfig: {
    responseModalities: ["AUDIO"],
    speechConfig: {
      voiceConfig: {
        prebuiltVoiceConfig: {
          voiceName: "Puck",
        },
      },
    },
  },
});

export class WebSpeechPlayer {
  private audioCtx: AudioContext | null = null;
  private nextStartTime: number = 0;

  private getAudioContext(): AudioContext {
    if (!this.audioCtx) {
      if (typeof window === "undefined") {
        throw new Error("AudioContext is only available in browser environments.");
      }
      const AudioContextClass =
        window.AudioContext ||
        (window as unknown as { webkitAudioContext: typeof AudioContext })
          .webkitAudioContext;
      this.audioCtx = new AudioContextClass({ sampleRate: 24000 });
    }
    return this.audioCtx;
  }

  async playSpeechStream(prompt: string) {
    const ctx = this.getAudioContext();
    if (ctx.state === "suspended") {
      await ctx.resume();
    }
    this.nextStartTime = ctx.currentTime;

    const responseStream = await model.generateContentStream(prompt);

    for await (const chunk of responseStream.stream) {
      const candidates = chunk.candidates || [];
      for (const candidate of candidates) {
        for (const part of candidate.content?.parts || []) {
          if ("inlineData" in part && part.inlineData?.data) {
            const rawPcm = this.base64ToArrayBuffer(part.inlineData.data);
            this.queuePcmChunk(rawPcm, ctx);
          }
        }
      }
    }
  }

  private queuePcmChunk(pcmData: ArrayBuffer, ctx: AudioContext) {
    const int16Array = new Int16Array(pcmData);
    const float32Array = new Float32Array(int16Array.length);

    // Convert 16-bit PCM integer samples to -1.0 .. 1.0 float samples
    for (let i = 0; i < int16Array.length; i++) {
      float32Array[i] = int16Array[i] / 32768.0;
    }

    const audioBuffer = ctx.createBuffer(1, float32Array.length, 24000);
    audioBuffer.copyToChannel(float32Array, 0);

    const source = ctx.createBufferSource();
    source.buffer = audioBuffer;
    source.connect(ctx.destination);

    const startTime = Math.max(this.nextStartTime, ctx.currentTime);
    source.start(startTime);
    this.nextStartTime = startTime + audioBuffer.duration;
  }

  private base64ToArrayBuffer(base64: string): ArrayBuffer {
    const binaryString = window.atob(base64);
    const len = binaryString.length;
    const bytes = new Uint8Array(len);
    for (let i = 0; i < len; i++) {
      bytes[i] = binaryString.charCodeAt(i);
    }
    return bytes.buffer;
  }
}
```

#### 2. Convert PCM Base64 to Playable WAV Blob for `<audio>` Elements

```typescript
export function pcmToWavBlob(pcmBytes: Uint8Array, sampleRate = 24000): Blob {
  const header = new ArrayBuffer(44);
  const view = new DataView(header);

  const numChannels = 1;
  const bitsPerSample = 16;
  const byteRate = (sampleRate * numChannels * bitsPerSample) / 8;
  const blockAlign = (numChannels * bitsPerSample) / 8;
  const dataSize = pcmBytes.byteLength;

  // RIFF identifier
  writeString(view, 0, "RIFF");
  view.setUint32(4, 36 + dataSize, true);
  writeString(view, 8, "WAVE");

  // fmt subchunk
  writeString(view, 12, "fmt ");
  view.setUint32(16, 16, true);             // Subchunk1Size
  view.setUint16(20, 1, true);              // AudioFormat (PCM = 1)
  view.setUint16(22, numChannels, true);
  view.setUint32(24, sampleRate, true);
  view.setUint32(28, byteRate, true);
  view.setUint16(32, blockAlign, true);
  view.setUint16(34, bitsPerSample, true);

  // data subchunk
  writeString(view, 36, "data");
  view.setUint32(40, dataSize, true);

  return new Blob([header, pcmBytes], { type: "audio/wav" });
}

function writeString(view: DataView, offset: number, string: string) {
  for (let i = 0; i < string.length; i++) {
    view.setUint8(offset + i, string.charCodeAt(i));
  }
}
```

______________________________________________________________________

## Best Practices & Troubleshooting

- **Audio Session & Permissions**: On iOS and Android, make sure the app's audio
  session/category is configured to allow playback (e.g.,
  `AVAudioSessionCategoryPlayback`).
- **Buffering & Jitter Prevention**: When streaming audio chunks via
  `generateContentStream`, queue buffers sequentially with exact timestamps or
  frame offsets to avoid audio stutter or gaps.
- **Model Compatibility**: TTS features require Gemini models with dedicated
  audio synthesis capabilities (`gemini-3.8-flash-tts` or
  `gemini-3.1-flash-tts-preview`). Do not use standard text-only model IDs.
- **App Check Protection**: Generating audio consumes quota. Protect your API
  endpoints by enforcing App Check on every client build.

# Firebase AI Logic on Android (Kotlin)

First, ensure you have initialized the Firebase App (see `firebase-basics`
skill). Then, initialize the AI Logic service as below

### 0. Enable Firebase AI Logic via CLI

Before adding dependencies in your app, make sure you enable the AI Logic
service in your Firebase Project using the Firebase CLI:

```bash
npx -y firebase-tools@latest init
# When prompted, select 'AI logic' to enable the Gemini API in your project.
```

______________________________________________________________________

### 1. Add Dependencies

In your module-level `build.gradle.kts` (usually `app/build.gradle.kts`), add
the dependency for Firebase AI:

```kotlin
dependencies {
    // [AGENT] Fetch the latest available BoM version from https://firebase.google.com/support/release-notes/android before adding this
    implementation(platform("com.google.firebase:firebase-bom:<latest_bom_version>"))

    // Add the dependency for the Firebase AI library
    implementation("com.google.firebase:firebase-ai")
}
```

______________________________________________________________________

### 2. Initialize and Generate Content

In your Activity or Fragment, initialize the `FirebaseAI` service and generate
content using a Gemini model:

```kotlin
import com.google.firebase.Firebase
import com.google.firebase.ai.ai
import com.google.firebase.ai.type.GenerativeBackend

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)

        // Initialize with replay protection using limited-use App Check tokens:
        val ai = Firebase.ai(
            backend = GenerativeBackend.googleAI(),
            useLimitedUseAppCheckTokens = true
        )

        // [AGENT] Replace "<latest_supported_model>" with the latest model from https://firebase.google.com/docs/ai-logic/models.md.txt
        val model = ai.generativeModel("<latest_supported_model>")

        // Generate content
        lifecycleScope.launch {
            try {
                val response = model.generateContent("Write a story about a magic backpack.")
                Log.d(TAG, "Response: ${response.text}")
            } catch (e: Exception) {
                Log.e(TAG, "Error generating content", e)
            }
        }
    }
}
```

#### App Check Replay Protection on Android

Generative and preview models enforce replay protection using short-lived
(5-minute) limited-use App Check tokens to prevent replay attacks. If you call a
model enforcing replay protection without limited-use tokens, the request is
rejected with:

```text
HTTP 403: "To access this model, you must enforce Firebase App Check"
```

To resolve this on Android, pass `useLimitedUseAppCheckTokens = true` when
calling `Firebase.ai`:

```kotlin
val ai = Firebase.ai(
    backend = GenerativeBackend.googleAI(),
    useLimitedUseAppCheckTokens = true
)
```

This ensures the SDK requests fresh limited-use tokens (via Play Integrity or
the debug provider) for each request instead of reusing cached tokens.

#### Jetpack Compose (Modern)

Initialize inside a `ComponentActivity` and use `setContent`:

```kotlin
import android.os.Bundle
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.lifecycle.lifecycleScope
import com.google.firebase.Firebase
import com.google.firebase.ai.ai
import kotlinx.coroutines.launch

class MainActivity : ComponentActivity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        val ai = Firebase.ai
        // [AGENT] Replace with the latest model from https://firebase.google.com/docs/ai-logic/models.md.txt
        val model = ai.generativeModel("<latest_supported_model>")
        
        lifecycleScope.launch {
            val response = model.generateContent("Hello Gemini!")
            setContent {
                MaterialTheme {
                    Text("AI Response: ${response.text}")
                }
            }
        }
    }
}
```

______________________________________________________________________

### 3. Multimodal Input (Text and Images)

Pass bitmap data along with text prompts:

```kotlin
val image1: Bitmap = ... // Load your bitmap
val image2: Bitmap = ...

val response = model.generateContent(
    content {
        image(image1)
        image(image2)
        text("Analyze these images for me. Compare these two items.")
    }
)
Log.d(TAG, response.text)
```

______________________________________________________________________

### 4. Chat Session (Multi-turn)

Maintain chat history automatically:

```kotlin
val chat = model.startChat(
    history = listOf(
        content("user") { text("Hello, I am a software engineer.") },
        content("model") { text("Hello! How can I help you today?") }
    )
)

lifecycleScope.launch {
    val response = chat.sendMessage("What should I learn next?")
    Log.d(TAG, response.text)
}
```

______________________________________________________________________

### 5. Streaming Responses

For faster display, stream the response:

```kotlin
lifecycleScope.launch {
    model.generateContentStream("Tell me a long story.")
        .collect { chunk ->
            print(chunk.text) // Update UI incrementally
        }
}
```

______________________________________________________________________

### 6. App Check (Debug Token Persistence)

When running on emulators or during development with App Check, persist a stable
debug token across emulator resets and fresh installs without hardcoding
secrets:

> [!WARNING] **CRITICAL: Never Hardcode or Commit Debug Tokens** Never hardcode
> debug token strings in `build.gradle.kts` or Kotlin source files. Store the
> token in gitignored `local.properties` and inject it dynamically.

1. In gitignored `local.properties`:

   ```properties
   APP_CHECK_DEBUG_TOKEN=<YOUR_DEBUG_TOKEN>
   ```

1. In `app/build.gradle.kts`:

   ```kotlin
   val localProperties = java.util.Properties().apply {
       val localPropertiesFile = rootProject.file("local.properties")
       if (localPropertiesFile.exists()) {
           load(localPropertiesFile.inputStream())
       }
   }
   val appCheckDebugToken = localProperties.getProperty("APP_CHECK_DEBUG_TOKEN")
       ?: System.getenv("APP_CHECK_DEBUG_TOKEN") ?: ""

   android {
       defaultConfig {
           // For normal emulator runs (injects into AndroidManifest)
           manifestPlaceholders["firebaseAppCheckDebugSecret"] = appCheckDebugToken

           // For instrumentation tests
           if (appCheckDebugToken.isNotEmpty()) {
               testInstrumentationRunnerArguments["firebaseAppCheckDebugSecret"] = appCheckDebugToken
           }
       }
   }
   ```

1. Initialize the debug provider in debug builds:

   ```kotlin
   if (BuildConfig.DEBUG) {
       Firebase.appCheck.installAppCheckProviderFactory(
           DebugAppCheckProviderFactory.getInstance()
       )
   }
   ```

# Firebase AI Logic Basics

## Initialization Pattern

You must initialize the ai-logic service after the main Firebase App.

```JavaScript
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend } from "firebase/ai";


// If running in Firebase App Hosting, you can skip Firebase Config and instead use:
// const app = initializeApp();

const firebaseConfig = {
  // ... your firebase config
};

const app = initializeApp(firebaseConfig);

// Initialize the AI Logic service (defaults to Gemini Developer API)
// Pass useLimitedUseAppCheckTokens: true to enable replay protection:
const ai = getAI(app, {
  backend: new GoogleAIBackend(),
  useLimitedUseAppCheckTokens: true,
});

const generationConfig = {
  candidateCount: 1,
  maxOutputTokens: 2048,
  stopSequences: [],
  temperature: 0.7,      // Balanced: creative but focused
  topP: 0.95,            // Standard: allows a wide range of probable tokens
  topK: 40,              // Standard: considers the top 40 tokens
};

// Specify the config as part of creating the `GenerativeModel` instance
// [AGENT] Replace "<latest_supported_model>" with the latest model from https://firebase.google.com/docs/ai-logic/models.md.txt
const model = getGenerativeModel(ai, { model: "<latest_supported_model>",  generationConfig });
```

### App Check Replay Protection

Generative and preview models enforce replay protection with 5-minute
limited-use App Check tokens. If you call a protected model without enabling
limited-use tokens, the request fails with:

```text
HTTP 403: "To access this model, you must enforce Firebase App Check"
```

To resolve this error on Web, initialize `getAI` with
`useLimitedUseAppCheckTokens: true`:

```javascript
const ai = getAI(app, {
  backend: new GoogleAIBackend(),
  useLimitedUseAppCheckTokens: true,
});
```

This instructs the SDK to fetch fresh, short-lived limited-use tokens (such as
via reCAPTCHA Enterprise or the debug provider) for requests rather than reusing
standard cached App Check tokens.

## Core Capabilities

Text-Only Generation

```JavaScript
async function generateText(prompt) {
  const result = await model.generateContent(prompt);
  const response = await result.response;
  return response.text();
}
```

## Multimodal (Text + Images/Audio/Video/PDF input)

Firebase AI Logic accepts Base64 encoded data or specific file references.

```JavaScript
// Helper to convert file to base64 generic object
async function fileToGenerativePart(file) {
  const base64EncodedDataPromise = new Promise((resolve) => {
    const reader = new FileReader();
    reader.onloadend = () => resolve(reader.result.split(',')[1]);
    reader.readAsDataURL(file);
  });
  
  return {
    inlineData: {
      data: await base64EncodedDataPromise,
      mimeType: file.type,
    },
  };
}

async function analyzeImage(prompt, imageFile) {
  const imagePart = await fileToGenerativePart(imageFile);
  const result = await model.generateContent([prompt, imagePart]);
  return result.response.text();
}
```

## Chat Session (Multi-turn)

Maintain history automatically using startChat.

```JavaScript
const chat = model.startChat({
  history: [
    {
      role: "user",
      parts: [{ text: "Hello, I am a developer." }],
    },
    {
      role: "model",
      parts: [{ text: "Great to meet you. How can I help with code?" }],
    },
  ],
});

async function sendMessage(msg) {
  const result = await chat.sendMessage(msg);
  return result.response.text();
}
```

## Streaming Responses

For real-time UI updates (like a typing effect).

```JavaScript
async function streamResponse(prompt) {
  const result = await model.generateContentStream(prompt);
  for await (const chunk of result.stream) {
    const chunkText = chunk.text();
    console.log("Stream chunk:", chunkText);
    // Update UI here
  }
}
```

Generate Images with Nano Banana

```Javascript
import { initializeApp } from "firebase/app";
import { getAI, getGenerativeModel, GoogleAIBackend, ResponseModality } from "firebase/ai";


// Initialize FirebaseApp
const firebaseApp = initializeApp(firebaseConfig);

// Initialize the Gemini Developer API backend service
const ai = getAI(firebaseApp, { backend: new GoogleAIBackend() });

// Create a `GenerativeModel` instance with a model that supports your use case
const model = getGenerativeModel(ai, {
  model: "<latest_supported_image_model>", // [AGENT] Replace with the latest image model from https://firebase.google.com/docs/ai-logic/models.md.txt
  // Configure the model to respond with text and images (required)
  generationConfig: {
    responseModalities: [ResponseModality.TEXT, ResponseModality.IMAGE],
  },
});

// Provide a text prompt instructing the model to generate an image
const prompt = 'Generate an image of the Eiffel Tower with fireworks in the background.';

// To generate an image, call `generateContent` with the text input
const result = await model.generateContent(prompt);

// Handle the generated image
try {
  const inlineDataParts = result.response.inlineDataParts();
  if (inlineDataParts?.[0]) {
    const image = inlineDataParts[0].inlineData;
    console.log(image.mimeType, image.data);
  }
} catch (err) {
  console.error('Prompt or candidate was blocked:', err);
}
```

## Advanced Features

Structured Output (JSON) Enforce a specific JSON schema for the response.

```JavaScript
import { getGenerativeModel, Schema } from "firebase/ai";
const jsonModel = getGenerativeModel(ai, {
    model: "<latest_supported_model>", // [AGENT] Replace with the latest model from https://firebase.google.com/docs/ai-logic/models.md.txt
    generationConfig: {
        responseMimeType: "application/json",
        // Optional: Define a schema
        responseSchema: Schema.object({ ... })
    }
});

async function getJsonData(prompt) {
    const result = await jsonModel.generateContent(prompt);
    return JSON.parse(result.response.text());
}
```

On-Device AI (Hybrid) Automatically switch between local Gemini Nano and cloud
models based on device capability.

```JavaScript
import {getGenerativeModel, InferenceMode } from "firebase/ai";

const hybridModel = getGenerativeModel(ai, { mode: InferenceMode.PREFER_ON_DEVICE });
```

## App Check (Debug Token Persistence)

When testing AI Logic on Web with App Check, avoid debug token churn when
clearing browser data or testing across private windows:

> [!WARNING] **CRITICAL: Never Hardcode or Commit Debug Tokens** Never hardcode
> debug token strings in web source code. Always load them from a local,
> gitignored environment file.

```javascript
import { initializeAppCheck, ReCaptchaEnterpriseProvider } from "firebase/app-check";

if (typeof self !== "undefined") {
  if (typeof process !== "undefined" && process.env.NODE_ENV === "development") {
    // ✅ SAFE: Load dynamically from Next.js environment variable; fallback to true to auto-generate
    self.FIREBASE_APPCHECK_DEBUG_TOKEN =
      process.env.NEXT_PUBLIC_APP_CHECK_DEBUG_TOKEN || true;
  } else if (typeof import.meta !== "undefined" && import.meta.env?.DEV) {
    // ✅ SAFE: Load dynamically from Vite environment variable; fallback to true to auto-generate
    self.FIREBASE_APPCHECK_DEBUG_TOKEN =
      import.meta.env.VITE_APPCHECK_DEBUG_TOKEN || true;
  }
}

const appCheck = initializeAppCheck(app, {
  provider: new ReCaptchaEnterpriseProvider("/* reCAPTCHA site key */"),
  isTokenAutoRefreshEnabled: true,
});
```

---
name: firebase-ai-logic-basics
description: Official skill for integrating Firebase AI Logic (Gemini API) into web applications. Covers setup, multimodal inference, structured output, and security.
version: 1.0.1
metadata:
  author: Google LLC
  category: AiAndMachineLearning
---

# Firebase AI Logic Basics

## Overview

Firebase AI Logic is a product of Firebase that allows developers to add gen AI
to their mobile and web apps using client-side SDKs. You can call Gemini models
directly from your app without managing a dedicated backend. Firebase AI Logic,
which was previously known as "Vertex AI for Firebase", represents the evolution
of Google's AI integration platform for mobile and web developers.

It supports the two Gemini API providers:

- **Gemini Developer API**: It has a free tier ideal for prototyping, and
  pay-as-you-go for production
- **Agent Platform Gemini API** (formerly branded Vertex AI): Ideal for scale
  with enterprise-grade production readiness, requires Blaze plan

Use the Gemini Developer API as a default, and only Agent Platform Gemini API
(formerly branded Vertex AI) if the application requires it.

## Setup & Initialization

### Prerequisites

- Before starting, ensure you have **Node.js 16+** and npm installed. Install
  them if they aren’t already available.
- Identify the platform the user is interested in building on prior to starting:
  Android, iOS, Flutter or Web.
- If their platform is unsupported, Direct the user to Firebase Docs to learn
  how to set up AI Logic for their application (share this link with the user
  https://firebase.google.com/docs/ai-logic/get-started)

### Installation

The library is part of the standard Firebase Web SDK.

`npm install firebase@latest`

If you're in a firebase directory (with a firebase.json) the currently selected
project will be marked with "current" using this command:

`npx -y firebase-tools@latest projects:list`

Ensure there's at least one app associated with the current project

`npx -y firebase-tools@latest apps:list`

Initialize AI logic SDK with the init command

`npx -y firebase-tools@latest init ailogic`

This will automatically enable the Gemini Developer API in the Firebase console.

More info in
[Firebase AI Logic Getting Started](https://firebase.google.com/docs/ai-logic/get-started.md.txt)

## Core Capabilities

> [!WARNING] **CRITICAL: Use current model names:** Always check the
> [Firebase AI Logic Models documentation](https://firebase.google.com/docs/ai-logic/models.md.txt)
> for the currently supported model names. Do NOT use `gemini-2.0-pro` or
> `gemini-2.0-flash` or other older models that are shutdown.

### Text-Only Generation

### Multimodal (Text + Images/Audio/Video/PDF input)

Firebase AI Logic allows Gemini models to analyze image files directly from your
app. This enables features like creating captions, answering questions about
images, detecting objects, and categorizing images. Beyond images, Gemini can
analyze other media types like audio, video, and PDFs by passing them as inline
data with their MIME type. For files larger than 20 megabytes (which can cause
HTTP 413 errors as inline data), store them in Cloud Storage for Firebase and
pass their URLs to the Gemini Developer API.

### Chat Session (Multi-turn)

Maintain history automatically using `startChat`.

### Streaming Responses

To improve the user experience by showing partial results as they arrive (like a
typing effect), use `generateContentStream` instead of `generateContent` for
faster display of results.

### Text-to-Speech (TTS) Generation

Generate spoken audio directly on client devices without a custom speech
backend. Firebase AI Logic supports speech synthesis using dedicated Gemini TTS
models:

- **Supported Models**: `gemini-3.8-flash-tts` and
  `gemini-3.1-flash-tts-preview`
- **Capabilities**:
  - Single-speaker voice persona selection (`voiceName`) and multi-speaker
    dialogues (up to 2 distinct speakers) via `SpeechConfig` /
    `MultiSpeakerVoiceConfig`
  - Direct audio responses via `responseModalities: [.audio]` / `["AUDIO"]`
  - Streaming speech responses with `generateContentStream` for low-latency
    playback
  - Audio directives (`[Audio Profile: ...]`, `[Scene: ...]`,
    `[Director's Note: ...]`) and emotional tags (`[whispers]`, `[laughs]`,
    `[slowly]`)
  - Client-side decoding of 24 kHz 16-bit linear PCM (`audio/l16`) via
    `AVAudioEngine` (iOS) / `AudioTrack` (Android) or WAV container format
    (`audio/x-wav`) via `AVAudioPlayer` (iOS) / `MediaPlayer` (Android)

See the [Text-to-Speech (TTS) Generation Guide](references/tts_generation.md)
for full implementation details, streaming patterns, and copy-paste code
snippets for iOS (Swift), Android (Kotlin), and Web.

### Generate Images with Nano Banana

> [!WARNING] **Use current Image model names:** Always check the
> [Firebase AI Logic Models documentation](https://firebase.google.com/docs/ai-logic/models.md.txt)
> for the currently supported image generation (Nano Banana) model names.

- Requires an upgraded Blaze pay-as-you-go billing plan.

### Search Grounding with the built in googleSearch tool

## Supported Platforms and Frameworks

Supported Platforms and Frameworks include Kotlin and Java for Android, Swift
for iOS, JavaScript for web apps, Dart for Flutter, and C Sharp for Unity.

## Advanced Features

### Structured Output (JSON)

Enforce a specific JSON schema for the response.

### On-Device AI (Hybrid)

Hybrid on-device inference for web apps, where the Firebase Javascript SDK
automatically checks for Gemini Nano's availability (after installation) and
switches between on-device or cloud-hosted prompt execution. This requires
specific steps to enable model usage in the Chrome browser, more info in the
[hybrid-on-device-inference documentation](https://firebase.google.com/docs/ai-logic/hybrid-on-device-inference.md.txt).

## Security & Production

### App Check

> [!WARNING] **Critical Safety Requirement:** In order to use AI Logic safely,
> you MUST set up App Check on your app. This prevents unauthorized clients from
> using your API quota and accessing your backend resources.

See
[App Check with reCAPTCHA Enterprise](https://firebase.google.com/docs/app-check/web/recaptcha-enterprise-provider.md.txt)
for setup instructions.

#### App Check Debug Tokens for Local Development & CI/CD

Because App Check attestation providers (like Play Integrity or DeviceCheck)
reject emulators, simulators, or CI environments, you must use **App Check Debug
Tokens** during development and testing to bypass standard attestation.

> [!WARNING] **CRITICAL: Never Hardcode or Commit Debug Tokens** App Check debug
> tokens allow clients to bypass attestation and access backend resources
> without a genuine device. Treat them as private secrets. **Never commit debug
> tokens to version control or hardcode raw token strings in client code** (Web,
> Android, iOS, or Flutter). Always inject them through local environment
> variables, gitignored local configurations, or CI secrets. If a token is
> compromised, revoke it immediately in the Firebase Console.

##### Local Development (Auto-Generated)

1. Configure your code's App Check provider to use the debug factory:
   - **Web**: Set `self.FIREBASE_APPCHECK_DEBUG_TOKEN = true;` before
     initializing App Check (generates a token in the browser console; do not
     hardcode secret strings in client code).
   - **Android**: Install `DebugAppCheckProviderFactory.getInstance()`.
   - **iOS**: Set provider factory to `AppCheckDebugProviderFactory()`.
1. Run your app in the emulator/localhost.
1. Look at your runtime debugger console / Logcat logs for the generated UUID:
   - *Example:* `AppCheck debug token: "123a4567-b89c-12d3-e456-789012345678"`
1. Register this token in the Firebase Console under **Security > App Check >
   Apps > Manage debug tokens**.

> **💡 iOS Tip (Prevent Debug Token Churn):** On iOS, simulator resets or fresh
> installs erase `NSUserDefaults`, causing the SDK to generate a new debug token
> UUID each time and invalidating tokens registered in the Firebase console. To
> persist a stable debug token without hardcoding secrets, set the
> `AppCheckDebugToken` environment variable in your unshared Xcode Scheme
> (**Edit Scheme -> Run -> Arguments -> Environment Variables -> Add
> `AppCheckDebugToken = <YOUR_DEBUG_TOKEN>`**). The iOS SDK automatically reads
> this environment variable at runtime. If setting it programmatically, load it
> dynamically from a gitignored local file rather than hardcoding the token
> literal into source code.
>
> **💡 Web Tip (Prevent Debug Token Churn):** Clearing site data, using
> incognito/private windows, or switching browsers generates a new debug token
> when `self.FIREBASE_APPCHECK_DEBUG_TOKEN = true` is used. To persist a stable
> debug token without leaking secrets into client bundles, load it from a
> gitignored local environment file (e.g., using
> `process.env.NEXT_PUBLIC_APP_CHECK_DEBUG_TOKEN` for Next.js or
> `import.meta.env.VITE_APPCHECK_DEBUG_TOKEN` for Vite). Never hardcode literal
> token strings into JavaScript or TypeScript source files.
>
> **💡 Android Tip (Prevent Debug Token Churn):** Emulator resets or clearing app
> storage erase `SharedPreferences`, causing `DebugAppCheckProviderFactory` to
> print a new debug token in Logcat. To keep a stable debug token across test
> runs and builds without committing it to git, store it in gitignored
> `local.properties` and inject it in `build.gradle.kts` if non-empty:
> `if (appCheckDebugToken.isNotEmpty()) { testInstrumentationRunnerArguments["firebaseAppCheckDebugSecret"] = appCheckDebugToken }`.

##### CI/CD Pipelines (Pre-Provisioned)

1. Generate and register a new debug token in the Firebase Console under
   **Security > App Check > Apps > Manage debug tokens**.
1. Add this token string as an encrypted secret in your CI system (e.g.
   `APP_CHECK_DEBUG_TOKEN`).
1. Configure your build to pass this secret as an environment variable to the
   SDK during test execution (e.g.
   `self.FIREBASE_APPCHECK_DEBUG_TOKEN = process.env.APP_CHECK_DEBUG_TOKEN`).

### Remote Config

Consider that you do not need to hardcode model names (e.g., a specific model
version string). Use Firebase Remote Config to update model versions dynamically
without deploying new client code. See
[Changing model names remotely](https://firebase.google.com/docs/ai-logic/change-model-name-remotely.md.txt)

> [!WARNING] **CRITICAL: Backend Provisioning Required** For all platforms
> (Flutter, Android, iOS, Web), you MUST run `npx firebase-tools init ailogic`
> to provision the service. `flutterfire configure` ONLY handles client
> configuration and does NOT enable the AI service, leading to
> `PERMISSION_DENIED` errors.

## Initialization Code References

- **Web Modular API**
  - Provider: Gemini Developer API
  - Reference: [usage_patterns_web.md](references/usage_patterns_web.md)
- **Android (Kotlin)**
  - Provider: Gemini Developer API
  - Reference: [usage_patterns_android.md](references/usage_patterns_android.md)
- **iOS (Swift)**
  - Provider: Gemini Developer API
  - Reference: [ios_setup.md](references/ios_setup.md)
- **Flutter (Dart)**
  - Provider: Gemini Developer API
  - Reference: [flutter_setup.md](references/flutter_setup.md)

> [!WARNING] **CRITICAL: Use current model names:** Always check the
> [Firebase AI Logic Models documentation](https://firebase.google.com/docs/ai-logic/models.md.txt)
> for the currently supported model names. Do NOT use `gemini-2.0-pro` or
> `gemini-2.0-flash` or other older models that are shutdown.

## References

[Web SDK code examples and usage patterns](references/usage_patterns_web.md)
[iOS SDK code examples and usage patterns](references/ios_setup.md)
[Flutter SDK code examples and usage patterns](references/flutter_setup.md)

[Android (Kotlin) SDK usage patterns](references/usage_patterns_android.md)
[Client-side Text-to-Speech (TTS) generation](references/tts_generation.md)