Page tree
Skip to end of metadata
Go to start of metadata

Business need

  • What problems are we trying to solve? Is the problem worth solving?
  • Who is the customer (business owner) of the solution who will make the final decision about the solution?
  • Who are the end-customers who will use the solution?
  • Does the problem require data-driven / machine learning solution or can we solve this by traditional software methods?
  • Are the expectations realistic?
  • Do we have the necessary human resources available? Data, model development and operations resources?
  • How will users be impacted by the data driven approaches? How these staff will be trained, monitored, and supported

Data

  • What data is needed to implement and train the model?
  • What data are we missing? What additional data is needed for proposed projects?
  • What is the data quality?
  • Is the data available and accessible?
  • Where and how will the data for model training be stored and managed during and after the project?
  • What are the possible constraints or challenges in accessing or incorporating this data?
  • What regulatory constraints exist on data collection, analysis, or implementation? Has the specific legislation and precedents been examined recently? What workarounds might exist?
  • What changes to data collection, coding, integration is needed?
  • Have we automated all the necessary data preparation steps?

Model training and development

  • What tools and libraries are used for to build the model?
  • What kind of models are suitable for the problem? Have we tried these models before?
  • What kind of development environment is suitable for developing the model? Local vs. cloud environments?
  • Does the model training require heavy processing? Special capacity / memory requirements?
  • How do you know whether a model is “good enough”? What metrics will be used to evaluate performance?
  • Have we automated all the necessary steps to be able to perform model re-training?

IT Implementation and integrations (deployment)

  • What IT systems will be used to support the model results and predictions?
  • What IT integration are required? How this IT integration will be done
  • What alternatives there are which may require less IT integration
  • Will IT systems need to be modified or developed to use the results of the project? Are there simpler implementations that could avoid substantial IT changes? If so, would this simplified implementation result in a significant reduction in impact?
  • What implementation challenges may occur
  • Which stakeholders will be needed to ensure implementation success? How might they perceive these projects and their potential impact on them?

Maintenance

  • When and how are the results and models transferred from development to operations teams? Are responsibilities clear?
  • How are the performance of models tracked? When does the team decide to rebuild models?
  • How do data scientists work with software engineers to ensure algorithms are correctly implemented?
  • When are test cases developed, and how are they maintained?
  • When is refactoring performed on code? How is the correctness and performance of models maintained and validated during refactoring?
  • How are maintenance and support requests logged? How are these requests handled?

Constraints

For each project being considered enumerate potential constraints that may impact the success of the project, e.g.:

  • What regulatory constraints exist on data collection, analysis, or implementation? Has the specific legislation and precedents been examined recently? What workarounds might exist?
  • What organizational constraints exist, including culture, skills, or structure?
  • What management constraints are there?
  • Are there any past analytic projects which may impact how the organization resources would view data-driven approaches?
  • No labels