Latest news
TutorialsMariGemma

Architectural Image Analysis with Gemma 4: Setup and Python Guide

Gemma 3 and later models can analyze images for tasks such as describing scenes, recognizing text, and locating objects. This guide covers the basics of trying Gemma 4 on architectural images with Python and Transformers.

Two interior visualizations being compared on an architecture team’s deskAI image
Representative image, generated with AI.Image: 3dsınıfı / FCA AI

In brief

  1. Gemma 3 and later models can be used to analyze visual content.
  2. The Gemma 4 example uses the image-text-to-text pipeline in the Transformers library.
  3. The guide covers scene description, analyzing multiple images, OCR, and object detection.
  4. Gemma 4 lets you choose an image token budget of 70, 140, 280, 560, or 1120.

What will you learn in this guide?

Gemma’s image-understanding capabilities can be used for more than simply describing an architectural image. Google’s guide notes that Gemma 3 and later models can describe image content, recognize objects, classify scenes, and draw inferences about visual context. It also provides examples of processing multiple images in a single request, recognizing text in images (OCR), and generating bounding-box coordinates for objects.

This walkthrough focuses on helping architecture and interior design teams describe project images at an early stage, examine images using specific questions, or list elements in an image. Treat the model’s output as an analysis aid that needs to be checked—not as a design decision or measured technical documentation. The source does not promise accuracy or architectural measurement precision.

Requirements

The example in the official guide uses the Transformers library alongside PyTorch and accelerate to install Python packages. The documentation specifies Transformers version 5.10.1 or later. The example model ID is google/gemma-4-E2B-it; a pipeline is set up with the image-text-to-text task for image and text inputs.

python
pip install torch accelerate
pip install 'transformers>=5.10.1'

This guide does not specify the hardware details required for installation and local execution. You should therefore check separately whether your computer is suitable in the environment where you plan to run the model. The example uses device_map='auto' and dtype='auto'; these are part of the pipeline configuration shown in the guide.

Steps

1. Set up the model pipeline

In the example below, MODEL_ID is set to the model ID provided in the guide. The pipeline is configured to process user messages containing both images and text.

python
MODEL_ID = 'google/gemma-4-E2B-it'
from transformers import pipeline
vqa_pipe = pipeline(
    task='image-text-to-text',
    model=MODEL_ID,
    device_map='auto',
    dtype='auto'
)

2. Ask a question about a project image

The source example loads an image from a URL and sends the image and text together in the message content. In your own work, replace img_url with an address that points to the image you want to analyze. The prompt below adapts the image-questioning method shown in the source for architectural review.

Prompt
Describe the space shown in this architectural image objectively. List the prominent structural elements, surfaces, openings, lighting conditions, and objects visible in the frame separately. Do not guess measurements or materials that cannot be verified from the image.

This prompt asks the model to describe the image in text and explicitly tells it not to present uncertain details as facts. Compare the generated description with the project drawings and material information.

python
from transformers import GenerationConfig
config = GenerationConfig.from_pretrained(MODEL_ID)
config.max_new_tokens = 512
gen_kwargs = dict(generation_config=config)
img_url = 'ANALIZ_EDILECEK_GORSELIN_URL_ADRESI'
messages = [
    {
        'role': 'user',
        'content': [
            {'type': 'image', 'url': img_url},
            {'type': 'text', 'text': 'Bu görselde görünen mekânı ve belirgin tasarım unsurlarını tarif et.'}
        ]
    }
]
output = vqa_pipe(messages, return_full_text=False, generate_kwargs=gen_kwargs)
print(output[0]['generated_text'])

The source example sets max_new_tokens to 512. This guide follows that example configuration; do not assume performance or quality outcomes beyond what the source states if you choose a different output length.

3. Compare two images in one request

The Gemma guide shows that you can provide multiple images in the same request and give them a shared text instruction. This approach could be used, for example, to compare two views of the same interior or to summarize the visible features of two alternative images separately. Do not assume the model will match every detail correctly.

Prompt
Examine the two images separately. Under a separate heading for each, summarize the visible spatial features and furniture layout. Mention only the common and different elements that can be seen in the images; do not interpret differences in angle or framing as design changes.

Add the two images to the message list as separate image items, then send the instruction as a text item. The guide’s example shows that multiple image items can be included in the same message; the architectural prompt here is a new example intended for practical use.

4. Read text in an image

The source demonstrates that the models can recognize multilingual text in images with an OCR example. You can try this feature to read a wayfinding sign, a visible label on a presentation board, or a sign in an interior photo. However, the source does not provide a specific accuracy rate.

Prompt
Transcribe all readable text in the image, briefly noting where each item appears. Mark characters you are unsure about as illegible rather than guessing. If there is no text in the image, say so.

When using text found in an image, check the letters and the context of the numbers against the original. For small text on presentation boards or low-resolution screenshots in particular, verify the output manually rather than copying it directly into a deliverable.

5. Object detection and resolution settings

The object-detection example in the guide expresses detected objects with labels and coordinates normalized to a 1000 × 1000 grid. It notes that these coordinates must be adapted to the actual image dimensions. So do not treat the output coordinates as pixel positions directly; the example in the source also scales the coordinates using the image’s width and height.

Gemma 4 models support variable resolution. The image token budget can be set to 70, 140, 280, 560, or 1120. The documentation explains that a higher budget can preserve more image detail, while a lower budget processes fewer patches and can increase inference speed. This is a trade-off between speed and visual detail; the source does not specify one ideal value for every project image. For detail-heavy tasks such as object detection, consider a higher budget and check the results at different budgets.

Common issues

Treating coordinates as pixels: According to the guide, bounding-box coordinates returned by object detection are normalized to a 1000 × 1000 scale. Without scaling them using the actual image width and height, the boxes may not be drawn in the correct positions.

Presenting details not visible in the image as facts: The material of a surface, the dimensions of a space, or the function of an unseen element cannot always be verified from an image. A safer working practice is to ask the model to mention only visible elements and verify the result against project information.

Overlooking differences in camera angle across multiple images: Two images may show the same space from different angles. In your comparison prompt, explicitly ask the model to consider each image separately and not to treat differences in framing as design changes.

Ignoring the token budget: Image resolution and the token budget affect how much visual detail is processed. Evaluating detail-sensitive detection with a lower budget may cause small elements to be missed; also bear in mind that a higher budget processes more patches, creating a speed-versus-detail trade-off.

Next steps: using it in architecture and visualization workflows

Architecture firms, interior designers, and archviz teams can try this method for supporting tasks such as preliminary review of image archives, scene descriptions, noting visible differences between two alternatives, or checking text in images. Students may also find it useful to have a reference image described and then compare the description with their own observations. These are possible applications of the documented image-understanding and multi-image processing capabilities, not a specific architectural product feature cited by the source.

For your next test, start with a single question, narrow the prompt to your project goal, and check the output against the image. If you need object detection, interpret the coordinate scale correctly; also consider the token budget, which affects the level of detail. The guide does not provide information about model licensing costs, specific hardware requirements, or compatibility with architectural software. Before making decisions on these points, check the conditions of the deployment and runtime environment you plan to use.

Source and license

This guide is adapted into Turkish from examples in Google’s Gemma image-understanding documentation. The source is provided under the CC BY 4.0 license. The architectural prompt examples were written specifically for this article.

Sources

1 source
A(
ai.google.dev (CC BY 4.0)ai.google.dev/gemma/docs/capabilities/vision/image
Summary

Source texts are not republished; short quotes are marked, everything else is our own summary and commentary.

3dsınıfı’s take
3dEditor’s assessment

For offices and students in Turkey, Gemma’s image-understanding capabilities can be tested for preliminary tasks such as quickly describing project images and reviewing references in an organized way. Teams that review large numbers of images at an early stage should use the output as a note to check, not as a design decision.

Before putting it into practice, it is important to verify the hardware requirements of the runtime environment and the licensing terms separately; the source does not provide clear information on these points. The token budget also involves a trade-off between speed and detail. Do not assume direct compatibility with architectural software, and check OCR results and object coordinates against the actual images.

Frequently asked questions

Can Gemma 4 analyze images?

Google’s guide says Gemma 3 and later models can be used to describe image content, recognize objects, understand scenes, and draw inferences from visual context.

Can Gemma 4 process multiple images in one request?

Yes. The guide shows that multiple images can be included in a single message and processed with a shared text instruction.

What image token budget options are available in Gemma 4?

The guide lists token budgets of 70, 140, 280, 560, and 1120. A higher budget can preserve more image detail, while a lower budget processes fewer patches.

Comments and the forum are in Turkish.Join the discussion
+

Related news