PKU-AIGI-500K

DCC 2024

JETCAS 2024

Abstract

In recent years, artificial intelligence-generated content (AIGC) enabled by foundation models has received increasing attention and is undergoing remarkable development. Text prompts can be elegantly translated/converted to photo-realistic high-quality images. This notable feature, however, has introduced extreme bandwidth requirements in compression and transmission of millions of AI-generated images (AIGI) for those AIGC services. Despite this, research on compression methods for AIGI is notably absent but undeniably necessary. This research addresses this critical gap by introducing the pioneering AIGI dataset, PKU-AIGI-500K, encompassing over 528k+ images and 105k+ diverse and comprehensive prompts derived from five major foundation models. Through this dataset, we delve into the exploration and analysis of the essential characteristics of AIGC images and empirically prove that existing data-driven lossy compression methods obtain sub-optimal or less efficient rate-distortion performance without fine-tuning, primarily due to a domain shift between AIGIs and natural ones. We thoroughly benchmark the rate-distortion performance and runtime complexity analysis of conventional and learned image coding solutions that are openly available, revealing new insights for emerging studies in AIGI compression. Moreover, to harness the full potential of redundant information in AIGI and its corresponding text, we propose an AIGI compression model (Cross-Attention Transformer Codec, CATC) trained on this dataset as a strong baseline. Subsequent experimental results demonstrate that our proposed model is able to achieve up to 30.09% bitrate reduction compared to the state-of-the-art (SOTA) H.266/VVC codec and outperforms the SOTA learned codec, which paves the way for future research in the field of AIGI compression.

(a) SD2.1B

there are many boxes of apples and other fruits on the street

there is a plate of food with broccoli and meat on it

zebras standing in a row under a shelter in a zoo

a lion cub stalks through a field of yellow flowers

roses and tulips in painting artist

the geothermal - hot river makes a mist in the mornings that comes through the trees

view on the roofs with church , a city

illustration with a temple and mountains .

(b) SD2.1

sun shining on the white mountain top

beauty is in the eye of the beholder .

the rapeseed has flowered but the gorgeous red of the poppy remains

the south western most tip .

there are many glasses of wine sitting on a table

arafed brick wall with clocks and a lamp on it

cute boy laughing in the park , close - up , slow motion

the beautiful strong dog walks outdoors alone in the winter afternoon stock photo - 15524665

there is a birthday cake with candles on it that says happy birthday harry

there is a red motorcycle parked on the side of the road

great spotted woodpecker isolated on a white background in studio shot

actor , film director and person attend premiere during festival

there is a teddy bear sitting in a rocking chair on the floor

the beautiful strong dog walks outdoors alone in the winter afternoon stock photo - 15524665

(d) MJ5.2

arafed collage of pictures of a plate of food including sausages, potatoes, and other foods

there is a clock tower in the middle of a park

there is a dog sitting on the ground with a frisbee in its mouth

there is a man that is talking on a cell phone

photo of invention along tourist attraction

woman in green shirt cutting up food in kitchen with red cabinets

(e) MOD

yellow flower with a blue sky background;
there are two sunflowers that are standing in the sun;
there is a sunflower that is blooming in the sun

there is a fish that is swimming in a tank;
there is a fish that is swimming in a tank;
there is a small fish that is looking at something

cars parked on the street in front of a row of multi - story buildings;
arafficial multicolored buildings line a street in a city;
araffes of cars parked on a street in front of a row of buildings

starry sky over the canyons and the milky;
milky over the canyons of bryce canyon, utah;
milky over the mountains and trees in the distance

there are two ducks that are standing together in the dirt;
there are two ducks that are under a large bird;
there is a bird that is sitting on the ground with its wings open

there is a black and white cow standing in a rocky field;
there is a bear that is walking through the rocky field;
there is a sheep that is standing in the grass by some rocks

BibTeX


        @inproceedings{duan2024neural,
          title={Neural Compression for AI Foundation Model Generated Images: Evaluation and Benchmark},
          author={Duan, Xunxu and Liu, Hongbin and Zhang, Li and Jia, Chuanmin},
          booktitle={2024 Data Compression Conference (DCC)},
          pages={553--553},
          year={2024},
          organization={IEEE}
        }


        @article{duan2024pku,
          title={PKU-AIGI-500K: A Neural Compression Benchmark And Model for AI-Generated Images},
          author={Duan, Xunxu and Ma, Siwei and Liu, Hongbin and Jia, Chuanmin},
          journal={IEEE Journal on Emerging and Selected Topics in Circuits and Systems},
          year={2024},
          publisher={IEEE}
        }

PKU-AIGI-500K

Abstract

Pipeline of dataset construction.

A comprehensive overview of the PKU-AIGI-500K database, includes the image resolution (in parentheses) and the quantity of images per category (in bold). For the SDXL1.0B category, both watermarked and non-watermarked test images are included.

Image Characteristics

(a) Hallucination

(b) Texture

(c) Color

CATC

More details please refer our paper.

Benckmark

More Examples

(a) SD2.1B

there are many boxes of apples and other fruits on the street

there is a plate of food with broccoli and meat on it

zebras standing in a row under a shelter in a zoo

a lion cub stalks through a field of yellow flowers

roses and tulips in painting artist

the geothermal - hot river makes a mist in the mornings that comes through the trees

view on the roofs with church , a city

illustration with a temple and mountains .

(b) SD2.1

sun shining on the white mountain top

beauty is in the eye of the beholder .

the rapeseed has flowered but the gorgeous red of the poppy remains

the south western most tip .

there are many glasses of wine sitting on a table

arafed brick wall with clocks and a lamp on it

cute boy laughing in the park , close - up , slow motion

the beautiful strong dog walks outdoors alone in the winter afternoon stock photo - 15524665

(c) SDXL1.0B

there is a birthday cake with candles on it that says happy birthday harry

there is a red motorcycle parked on the side of the road

great spotted woodpecker isolated on a white background in studio shot

actor , film director and person attend premiere during festival

there is a teddy bear sitting in a rocking chair on the floor

the beautiful strong dog walks outdoors alone in the winter afternoon stock photo - 15524665

(d) MJ5.2

arafed collage of pictures of a plate of food including sausages, potatoes, and other foods

there is a clock tower in the middle of a park

there is a dog sitting on the ground with a frisbee in its mouth

there is a man that is talking on a cell phone

photo of invention along tourist attraction

woman in green shirt cutting up food in kitchen with red cabinets

(e) MOD

yellow flower with a blue sky background; there are two sunflowers that are standing in the sun; there is a sunflower that is blooming in the sun

there is a fish that is swimming in a tank; there is a fish that is swimming in a tank; there is a small fish that is looking at something

cars parked on the street in front of a row of multi - story buildings; arafficial multicolored buildings line a street in a city; araffes of cars parked on a street in front of a row of buildings

starry sky over the canyons and the milky; milky over the canyons of bryce canyon, utah; milky over the mountains and trees in the distance

there are two ducks that are standing together in the dirt; there are two ducks that are under a large bird; there is a bird that is sitting on the ground with its wings open

there is a black and white cow standing in a rocky field; there is a bear that is walking through the rocky field; there is a sheep that is standing in the grass by some rocks

BibTeX

yellow flower with a blue sky background;
there are two sunflowers that are standing in the sun;
there is a sunflower that is blooming in the sun

there is a fish that is swimming in a tank;
there is a fish that is swimming in a tank;
there is a small fish that is looking at something

cars parked on the street in front of a row of multi - story buildings;
arafficial multicolored buildings line a street in a city;
araffes of cars parked on a street in front of a row of buildings

starry sky over the canyons and the milky;
milky over the canyons of bryce canyon, utah;
milky over the mountains and trees in the distance

there are two ducks that are standing together in the dirt;
there are two ducks that are under a large bird;
there is a bird that is sitting on the ground with its wings open

there is a black and white cow standing in a rocky field;
there is a bear that is walking through the rocky field;
there is a sheep that is standing in the grass by some rocks