AI Visual Quality Annotation Specialist

Canva

Canva

Software Engineering, Data Science, Quality Assurance
Beijing, China
Posted on Dec 16, 2025

Job Description

We’re seeking an AI-Native Visual Designer. A new-era hybrid designer with deep visual judgment, strong design fundamentals, and hands-on experience shaping AI systems. You will help train, evaluate, and improve AI-powered creative tools by applying expert annotation, quality assessment, and design-led thinking. This role sits at the intersection of design, data, and product/engineering, ensuring our models deliver visually excellent, brand-aligned, and human-centred output.

1. Visual Evaluation & Annotation

  • Annotate and label large volumes of visual data (images, UI layouts, compositions, graphics, photos).

  • Provide high-quality tagging related to style, composition, colour theory, typography, and visual hierarchy.

  • Evaluate AI-generated output for accuracy, aesthetic quality, usability, and alignment with design principles.

  • Identify patterns, errors, and edge cases to improve model performance.

2. Design Quality Assessment

  • Develop visual scoring criteria and rubrics to guide model improvement.

  • Review outputs across various creative formats (illustration, branding, UI components, marketing visuals).

  • Validate whether outputs meet user expectations, brand standards, accessibility guidelines, and ethical guidelines.

3. Cross-Functional Collaboration

  • Work closely with machine learning engineers, research scientists, and product teams to provide design-driven feedback loops.

  • Translate design concepts into structured feedback that model teams can use (e.g., style attributes, visual defects, composition rules).

  • Help communicate creative insights into technical documentation.

4. Prototyping & Creative Exploration

  • Use generative AI tools to create visuals, test hypotheses, and prototype user workflows.

  • Explore new capabilities and push the boundaries of AI-assisted creativity.

  • Offer creative direction for improving model outputs and user-facing features.

5. Data & Process Stewardship

  • Maintain data quality and consistency in annotation projects.

  • Contribute to the development of taxonomies for visual styles, attributes, and design patterns.

  • Document annotation methodologies and participate in improving annotation tooling.

Qualifications

Design Foundations

  • Strong grasp of composition, color theory, typography, layout, visual hierarchy, and storytelling principles.

  • Portfolio demonstrating visual design excellence across one or more domains (graphic, product, UI/UX, illustration, etc.)

AI & Technical Understanding

  • Familiarity with generative AI tools (image models, large language models, design AIs).

  • Understanding of how AI training data, annotation, and quality metrics influence model behavior.

  • Ability to evaluate model output with precision and articulate actionable feedback.

Collaboration & Communication

  • Able to translate design critique into structured, repeatable feedback for technical and product partners.

  • Experience collaborating with engineers, researchers, or technical teams is a plus.

  • Strong written communication skills and comfort in documentation-heavy workflows.

Bonus Skills

  • Experience with Figma, Adobe Creative Cloud, or 3D tools.

  • Prior data labeling, annotation, or ML operations experience.

  • Basic familiarity with scripting (Python), prompt engineering, or dataset tooling.

  • Knowledge of design accessibility (WCAG), usability heuristics, or user research methods.

This role is perfect for designers who are:

  • Curious about the future of AI-driven creativity

  • Comfortable with both artistic judgment and structured evaluation

  • Excited to influence how AI models “see” and “create”

  • Skilled in both visual taste and systematic thinking