Transforming the Data Center - Scaling Computing Infrastructure Sustainably.pdf

1、 2024 ArmEddie RamirezVP of Marketing,Infrastructure,ArmTransforming the Data Center Scaling Computing Infrastructure Sustainably 2024 Arm 2024 ArmAI is Everywhere 2024 Arm 2024 Arm50%of companies with 5,000employees were using AI$50B VCfunding in AI enterprises and startups in 2023AI Server Market

2、to Reach$150 Billion by2027 Liu Yangwei,chairman of Foxconn;Source:AnandTechSource:CrunchbaseSurvey:NBER Paper-AI Adoption In America-Oct 2023 2024 Arm 2024 ArmAI is Expensive 2024 Arm 2024 Arm5Image ClassificationText GenerationImage GenerationCarbon Footprint of AI400 x7xInference energy(kWh)2024

3、Arm 2024 Arm 2024 Arm6 2024 ArmCarbon Footprint of AI*Est energy per 1,000 queriesAI TaskInference energy(kWh)*Text generation0.047 kWhImage generation(Multi Modal)2.907 kWhUsageTotal Est Energy10 Trillion Tokens47 Billion kWh25 Billion Results72.6 Billion kWhSource:Tirias Research;BCG Analysis;Arxi

4、v.org Power Consumption/Year of Portugal(48.4 B kWh)2024 ArmWorkload SpecificGPU/NPUGeneral PurposeCPUInfrastructureDPU/IPUG R A C E H O P P E RG R A C E H O P P E RB L U E F I E L DB L U E F I E L DT R A I N I U M 2T R A I N I U M 2G R A V I T O N 4G R A V I T O N 4N I T R ON I T R OM A I A 1 0 0M

5、A I A 1 0 0C O B A L T 1 0 0C O B A L T 1 0 0A Z U R E B O O S TA Z U R E B O O S TCompute Innovation in the AI EraCustom Silicon Co-designed for AIIncludes both Arm and non-Arm based designsC L O U D T P UC L O U D T P UE 2 0 0 0 I P UE 2 0 0 0 I P UA X I O N C P UA X I O N C P U 2024 ArmCost,laten

6、cyPerf,batch-size$,real-time$,offline100s,1-101000s,100s*Above chart is based on LLaMa2-7B performance other models have similar characteristics,but different crossover points.GPU Inference CPU Inference TCO Analysis of Inferencing on CPU vs GPU 2024 ArmGenerative AI on Arm Neoverse150200250300Token