Top suggestions for New AI TechnologiesExplore more searches like New AI Technologies |
Explore more searches like
NewAI Technologies
+technology+is+revolutionizing+the+computing+industry.+Not+only+its+applications+have+broadened+to+various+sectors+but+also+poses+new+system+design+and+optimization+opportunities.+The+technology+is+capable+of+understanding+and+responding+in+multiple+modalities.+However%252C+the+advanced+capability+currently+comes+with+significant+system+resource+demands.+To+sustainably+scale+generative+AI+capabilities+to+billions+of+users+in+the+world%252C+inference+must+be+fast+and+efficient.+This+paper+pinpoints+key+system+design+and+optimization+opportunities+by+characterizing+a+family+of+emerging+multi-modal+generation+models+on+real+systems.+Auto-regressive+token+generation+is+a+critical+latency+performance+bottleneck%252C+typically+dominated+by+GPU+idle+time.+In+addition+to+memory-intensive+attention+across+the+generative+AI+models%252C+linear+operations+constitute+significant+inference+latency+due+to+the+feed+forward+networks+in+Transformer-based+models.+We+demonstrate+that+state-of-the-art+optimization+levers%252C+spanning+from+applications+to+system+software+and+hardware%252C+set+a+3.88x+better+baseline.&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.b52e13ad6582a9f4185b1553f9438989%3frik%3dezwMRfWdBJ0l2g%26pid%3dImgRaw%26r%3d0&exph=630&expw=1200&q=New+AI+Technologies&FORM=IRPRST&ck=94C21C0DCD3F7FAEE7759556CBA41117&selectedIndex=29&itb=0)
Some results have been hidden because they may be inaccessible to you.Show inaccessible results
Top suggestions for New AI TechnologiesExplore more searches like New AI Technologies |
