Immuta：使用自动数据访问控制构建端到端 MLOps 工作流.pdf

1、Building an End-to-End MLOps Workflow with Automated Data Access ControlsDatabricks2023AgendaBuilding an End-to-End MLOps Workflow with Automated Data Access ControlsData and MLOps Approach at WorldQuant PredictivePutting Data at the Center-Data Access Controls with ImmutaPutting it all together at

2、WorldQuant PredictiveWorldQuant Predictive IntroductionSignal Factory finds predictive signals in the data with networks of ensemble models.WorldQuant Predictive delivers ready-made predictive AI solutions,trained on the worlds data.New York,NY Global Team of Data Scientists&EngineersGlobal Research

3、 NetworkExpands our expertiseWhat We DoWho We AreWe scout&curate differentiated data,which are derived from public,commercial and non-traditional sources.Quanto,our AI platform,enables anyone to access the models to immediately predict outcomes,simulate scenarios and optimize decisions.WorldQuant Pr

4、edictive Data and MLOps Workflows OverviewData EngineersData AnalystsData ScientistsML Ops Business SMEsCustomersPersonas,Use Cases,Tech StackData IngestionData QualityData ExplorationData Governance/SecurityFeature ExtractionML Model ExperimentationML Testing/ValidationML DeploymentDatabricksSnowfl

5、ake+SnowparkImmutaMLFlowAirflowDBTGitWQPs BigFeat PersonasUse CasesTech Stack(Our Toolkit)Law of Conservation of Complexity:Removing complexity from user experience moves it to system setupOur ApproachWe want to hide complexity of environments and tools from usersKeep it simple-use consistent toolki

6、t and building blocksEverything as code-use Git as promotion mechanismTrust in policies to provide appropriate data when neededWorldQuant Predictive Data and MLOps Workflows OverviewIngest through Databricks+ImmutaTransform with DBTExplore data with SnowflakeTrain in DatabricksStore models in MLFlow