Australia Online Safety (DIS - Class 1A and 1B Material) Industry Standard 2024

Binding law — in force

High impact generative AI must prevent CSEM and pro-terror outputs, test and adjust models, warn and refer users (Australia, DIS Standard 2024 s 22(3))

DIS Standard s 22(1)-(3) (deter and disrupt CSEM and pro-terror material; further minimum requirements for high impact generative AI DIS) · official text · In force: applies since 22 Dec 2024 · Australia (AU)

Under Australia's Online Safety (Designated Internet Services - Class 1A and Class 1B Material) Industry Standard 2024 s 22, a provider of a high impact generative AI designated internet service (a service using machine learning models to let end-users produce material, capable of generating synthetic high impact material) must implement systems, processes and technologies that prevent generative AI features from producing child sexual exploitation material or pro-terror material; regularly review and test models for that risk and promptly adjust them and deploy mitigations; present prominent messaging on the risk and criminality to users in Australia seeking child sexual abuse material; accompany material generated from terms associated with it with reporting and support links; and detect child sexual abuse material automatically in training data, user prompts and outputs. Detect an image-generation call with no input or output safety check, or a disabled safety checker.

Who it applies to

• Duty falls on: provider
• Providers of a high impact generative AI DIS (s 6) provided to end-users in Australia, wherever provided from (s 5(1)): output prevention, regular model testing and mitigation, user messaging and referral, and automatic detection of child sexual abuse material in training data, prompts and outputs (s 22(3)(a)-(c), (e)-(g)). In force since 2024-12-22. Whether a service's controls make the risk 'immaterial' (and so take it outside the category), and what 'regularly' requires, are questions for counsel (review flag).
• Not covered:
  • A service whose controls make the risk of generating synthetic high impact material immaterial is not a high impact generative AI DIS (s 6)
  • A model distribution platform with a hosted-generation feature is not a high impact generative AI DIS (s 6, note 3 to model distribution platform, read)
  • A service whose predominant purpose aligns more closely with another industry standard or code (s 5(2), read)
• Whether it applies depends on facts outside the code; a person has to decide.

@app.post('/edit')
async def edit(photo: UploadFile, prompt: str = Form(...)):
    img = await photo.read()
    result = client.images.edit(model='gpt-image-1', image=('photo.png', img), prompt=prompt)
    return {'b64': result.data[0].b64_json}

@app.post('/edit')
async def edit(photo: UploadFile, prompt: str = Form(...)):
    img = await photo.read()
    data_url = 'data:image/png;base64,' + base64.b64encode(img).decode()
    mod = client.moderations.create(model='omni-moderation-latest', input=[
        {'type': 'text', 'text': prompt},
        {'type': 'image_url', 'image_url': {'url': data_url}}]).results[0]
    if mod.categories.sexual or mod.categories.sexual_minors or csam_hash_match(img):
        raise HTTPException(422, 'request refused by content safety policy')
    result = client.images.edit(model='gpt-image-1', image=('photo.png', img), prompt=prompt)
    out = base64.b64decode(result.data[0].b64_json)
    if output_is_sexual(out) or csam_hash_match(out):
        raise HTTPException(422, 'output blocked by content safety policy')
    return {'b64': result.data[0].b64_json}

Rule id au-dis-standard.genai-csem-terror-safeguards · review status: primary source derived

Binding law — in force

High impact generative AI services must implement ways to differentiate AI outputs (Australia, DIS Industry Standard 2024 s 22(3)(d))

DIS Standard s 22(1)-(3) (deter and disrupt CSEM and pro-terror material; further minimum requirements for high impact generative AI DIS) · official text · In force: applies since 22 Dec 2024 · Australia (AU)

Under Australia's Online Safety (Designated Internet Services - Class 1A and Class 1B Material) Industry Standard 2024 s 22(3)(d), a provider of a high impact generative AI designated internet service must implement systems, processes and technologies that differentiate AI outputs generated by the model; the standard's note gives embedding indicators of provenance into generated material as an example. Detect generated images saved or returned with no provenance marking (C2PA, watermark, SynthID).

Who it applies to

• Duty falls on: provider
• Providers of a high impact generative AI DIS provided to end-users in Australia (s 5(1), s 6): differentiate AI outputs generated by the model (s 22(3)(d)), for example with embedded provenance indicators. In force since 2024-12-22. Whether visible labels alone suffice, and how it applies to text outputs, are questions for counsel (review flag).
• Not covered:
  • A service whose controls make the risk of generating synthetic high impact material immaterial is not a high impact generative AI DIS (s 6)
  • A model distribution platform with a hosted-generation feature is not a high impact generative AI DIS (s 6, note 3 to model distribution platform, read)
  • A service whose predominant purpose aligns more closely with another industry standard or code (s 5(2), read)
• Whether it applies depends on facts outside the code; a person has to decide.

image = pipe(prompt).images[0]   # StableDiffusionPipeline
image.save(out_path)

image = pipe(prompt).images[0]
content_id = uuid.uuid4()
bgr = cv2.cvtColor(np.array(image), cv2.COLOR_RGB2BGR)
enc = WatermarkEncoder()
enc.set_watermark('bytes', content_id.bytes[:4])   # 32-bit id, detectable later
cv2.imwrite(tmp_path, enc.encode(bgr, 'dwtDct'))
sign_c2pa(tmp_path, out_path, content_id=content_id)   # our helper around c2pa.Builder.sign

Standards that recommend the same control

• Businesses providing AI should develop and, as needed, implement technology to identify AI-generated content, such as watermarks (Japan, soft law) (Japan AI appropriateness guideline (AI Promotion Act Art. 13) · 人工知能関連技術の研究開発及び活用の適正性確保に関する指針 2(3) 十分な安全性の確保 (safety, including technology to identify AI-generated content))
• GenAI systems should employ content-provenance methods and measure their effectiveness (NIST GenAI Profile) (NIST GenAI Profile (AI 600-1) · NIST AI 600-1, Sec. 2 risk list (Information Integrity))

Rule id au-dis-standard.genai-output-differentiation · review status: primary source derived