修宇亮-China3DV.pptx

1、Feat2GSProbing Visual Foundation Models with Gaussian Splatting,Westlake UniversityMax Planck Institute for Intelligent SystemsUniversity of Tbingen,Tbingen AI CenterMax Planck Institute for Informatics,How well do they understand the 3D world?,Visual Foundation Models,We need 3D probing.,Previous 3

2、D Probing,Image,Features,Features,Image,3DGS,Novel view synthesis,2D ground-truth,Depth/normal,3D ground-truth,Evaluation,New Benchmark,Feat2GS as Probe,Gaussian decoder,Gaussiandecoder,Splatting,CLIP,DINO,MAST3R,Novel View Synthesis,shared weight,Novel View Synthesis,Probing Schemesc,Pipeline,Novel

3、 View Synthesis,Probing Schemes,Pipeline,DINOv2,DUSt3R,Accuracy:-2.448Completeness:-.277Distance:-6.163,PSNR:+1.02SSIM:+.0233LPIPS:-.0301,VS.,Our Findings:3D Metrics and 2D Metrics are well-aligned.,Pointcloud Error Map,Novel View Synthesis,Stable Diffusion,RADIO,VS.,Our Findings:3D Metrics and 2D M

4、etrics are well-aligned.,Pointcloud Error Map,Novel View Synthesis,Accuracy:-1.68Completeness:-.092Distance:-3.625,PSNR:+.61SSIM:+.0123LPIPS:-.0178,Geometry Probing,Geometry Probing,Texture Probing,Training views,Findings:Foundation models capture geometry well,but struggle with texture.,Application

5、,Novel View Synthesis/Normal Results,Input images,Welcome to our Website!fanegg.github.io/Feat2GSPlay online and interact now!,Interactive Probing Results,Easi3REstimating Disentangled Motion from DUSt3R Without Training,Westlake UniversityMax Planck Institute for Intelligent SystemsUniversity of Tbingen,Tbingen AI Center,ETCHGeneralizing Body Fitting to Clothed Humans via Equivariant Tightness,Westlake UniversityMax Planck Institute for Intelligent SystemsBerkeley AI Research(BAIR),Zeyu Cai1,Michael J.Black2,远兮实验室,endless.do,