Smith College course (https://smithcollege-sds.github.io/capstone/)
description slides
(https://beanumber.github.io/talks/sdss2020/capstone.html#11) mention
"USRESP-style final paper" and an "Ethics" component. Here is more on
USRESP: https://www.causeweb.org/usproc/usresp
Click here to reveal more content
This text is hidden by default and unhides when you click the header!
By the end of this course, students will be able to:
Rapid Problem Scoping & Refinement via Hypothetical, End-to-End Analysis Pipelines
Understanding the client’s challenge and developing a design document with:
Background: sufficient information to understand the client’s question/challenge and the "statistical questions”
Statistical Question: Precise statistical question(s) that explicitly specify the target population of interest and what is precisely being measured
Units: What are the elements of the population that are being studied? What information uniquely identifies a unit? Are there any blocks that need to be considered?
Treatments: What interventions or possible interventions are being applied to a unit? What information uniquely defines an intervention? Are there treatment factors that need to be considered?
Measurements: What exactly are we going to measure for each unit and treatment? A data manifest (variable names, dictionary, value types and ranges). How is the data going to be stored?
Analysis: Assuming the measurements can be collected, how much data is needed and why? How is the data going to be analyzed? What are the graphs, figures, tables, summaries to be generated and how do they address the statistical question(s)?
Hypothetical, End-to-end Pipelines with Synthetic Data Sets. For each data set (at least two that produce two opposite/complimentary conclusions) create a notebook with
A background section (same as above)
Statistical question (same as above)
Dependencies required and why (then include/import the dependencies)
Units and their simulation manifest
Treatments and their simulation manifest
Measurements and their simulation manifest
Execution of the analysis with figures, tables, summaries that are integrated into a narrative with conclusions that answer the statistical question(s).
Sensitivity analysis: how do your conclusions agree/disagree (magnitude and direction) if you alter certain choices in the data analysis process? (E.g., different assumptions, different models, different sample sizes, different tests, different pre-processing)
A discussion of the consequences for the client’s problem
Using the end-to-end pipelines, collaborate with the client to suggest refined project scope, co-design key deliverables, and map it to the final schedule. This should be plan that specifies:
The content of the mid-semester presentation and report, and how success will be measured for the presentation and report.
The content of end-of-semester presentation, report and hand-off package, and how success will be measured for the presentation, report and hand-off package.
Engineering professional-grade, reproducible, client-centric data science workflows.
Configuring and working with data science tools to support collaboration, development, documentation, analysis and reporting.
Configuring an Independent Development Environment (e.g., VS Code) for development support (e.g., verification, syntax highlighting, style adherence), analysis and reporting.
Configuring and leveraging interactive notebooks (such as RMarkdown and Jupyter Notebooks) for principled, understandable prototypes of analysis pipelines
Understanding and managing SSH keys for secure remote access, and GPG keys for signing and verifying digital work.
Mastering core commands of version control systems (e.g., Git) (e.g., init, status, add, commit, branch, switch, merge, diff, revert) and healthy version control practices (e.g., regular, informative commits)
Engaging in asynchronous collaboration using version control tools such as pull, push, and remote; and pull requests for code review and collaborative refinement.
Leveraging automated workflows (e.g., GitHub actions) to ensure code integrity (e.g., unit tests passing and code coverage), document building, and other analysis pipeline execution checks.
Architecting professional-grade, client-centric analysis pipelines with modular logic and unambiguous entry points
Consolidating client-relevant variables (e.g, hyperparameters, file paths, global flags) into a single configuration file (e.g., config.yaml)
Planning and creating file structures that logically group functions and correspond to a high-level abstraction of data flow.
Using good software development practices such as docstrings, line comments, unit tests, and code coverage to ensure analysis quality.
Specifying clear entry points (e.g., start script) and exit points (final outputs such as figures, tables, logs).
Developing comprehensive hand-off package that ensures a project can be understood, reproduced and extended by clients
Explaining the architectural and design choices underlying the hand-off package for clients to understand possible trade-offs, benefits and limitations of the hand-off package
Creating comprehensive documentation that includes
A reproducibility guide and tutorial to enable a client to set up the environment and run the analysis pipeline with possible changes to the data and configuration file;
A statistical manual that describes and justifies the design, analysis pipeline logic, sensitivity analysis, and outputs that support the conclusions
A software manual that describes and justifies system requirements and software design; and describes code, algorithms, interfaces and APIs
Writing configuration files for container images (e.g., Dockerfiles) that explicitly declare the base operating system, system libraries and language dependencies required for the pipeline in order to package the analysis pipeline into a portable, isolated environment
Building, testing and running container images locally and remotely to verify that execution is consistent and self-contained.
Executing computational workflows on remote, large-scale, or high-performance computing infrastructure
Configuring jobs for batch processing systems and cluster schedulers to manage resource allocation
Transferring, synchronizing, and tracking datasets, intermediate data transformations, and other data artifacts securely between local environments and remote services (e.g., UW Research Data Services).
Adapting local pipeline execution to run seamlessly across multiple platforms such as CHTC or cloud-based computing environment (e.g., AWS, Google Cloud)
(schedule)
Ideas:
For meeting next week:
- teach students how to use video conferencing in, e.g., MH 6683,
where Dan and Vivak were for our meeting with John (Zoom) and Derek
(Zoom), and which worked impressively well compared to Zoom
-
Write example course project with all deliverables. Share
with students to help define expectations.
-
Students write their own milestones and assessments,
e.g. "Click on ___ and this analysis runs: ___, producing
this output: ___"; these are revised and extended several
times through the course
-
Guest lectures:
-
Technical skills:
- R Markdown, Jupyter Lab, Latex
- Positron (a new IDE from Hadley Wickham)
- git/GitHub
- container and reproducible research
(docker
tutorial)
- remote services, including CHTC
- maybe commercial cloud computing, e.g. AWS or Google Cloud
- UW Research Data Services
- How to make presentation slides:
Maybe students can research this?
-
UW training:
- Stewardship
- Research security
-
Assign teams after surveying interest in client proposals.
- Students drive and write most elements--clients don't drive project.
- Students deliver slides for each weekly client meeting (or
each of several milestone meetings); practice in group-pairs
- Refine scope of project:
- Question
- Feasible
- Deliverable
- Assessable
- Communicable
- Valuable to client
- Rapidly iterate on data analysis to refine project scope;
document what is tried, rejected, and retained for development.
- simulate data from manifest, write analysis pipeline to
concretize mental models
- Contextualize statistical analysis:
- assumptions
- statistical validity
- weakness
- how to resolve holes in future
- Maybe we can learn from this course: UW MS-Information Capstone
- Derek found this UF course and syllabus (whose p. 3 grading scheme uses "E" instead of "F"):
- Consider schedule on syllabus p. 6: It teaches:
- Writing technical reports and literature reviews
- Intro STAT topics
- Planning a speech and creating effective slides
- Presenting statistical results to lay audiences
- Preparing and presenting research posters
- Require criticising peers' work
Strategic Communication: Adapt technical findings for diverse
audiences, moving between deep-dive technical reports and high-level
executive briefings.
With whom am I communicating?
- self:
- learn about process, time management, priority-setting;
self-acceptance; fail repeatedly and gracefully, recognizing
opportunity revealed by failure (maybe quote Elon Musk on
failure?)
- peers:
- informal team discussion in class
- practice presentations to rest of class
- client:
- via web conference:
- using Zoom
- using Morgridge conference room
- broad audience:
- in person for final on-campus presentation
In what format am I communicating?
- Talking:
- informal
- formal presentation to peers
- unsupported by slides or writing on board
- supported by slides (this overlaps with "Writing" below)
- presentation to broader audience
- body language do's and don'ts
- We can seek help on "How to talk to people" from WSB or Theatre
(maybe Colleen Conroy?).
- Writing
- text
- email
- slides (this overlaps with "Talking" above)
- data science report
- executive one-pager
- We can seek help from Writing Center (I requested help Tu 6/16/26), e.g.:
- Academic and Professional Writing, writing.wisc.edu/handbook/assignments
- An Overview of the Writing Center for Instructors, writing.wisc.edu/instructor
- Collaborate with the Writing Center, writing.wisc.edu/instructor/outreach
- Writing Across the Curriculum (WAC), wac.wisc.edu
- Services for Students, writing.wisc.edu/wp-content/uploads/sites/535/2022/01/Writing-Center-services-flier.pdf
- Writing (a second time, with more details)
- text:
- short greeting, "Hi ___."
- if first message, give your name and organization
- 1 or 2 sentences including easy question or call to action
- correct spelling and grammar
- business hours only
- use email for long conversation
(examples: textus.com/blog/anatomy-good-business-text-message)
- email:
- dense short "Subject:"
- greeting: "Dear ___" or "Hello ___" or "Hi ___"
- short body with short paragraphs
- closing: "Sincerely, ___", etc.
- professional tone
- limit use of "Reply all"
- proofread
- ...
- (Writing Center has page at
writing.wisc.edu/handbook/assignments/advice-for-students-writing-a-professional-email)
- slides (this overlaps with "Talking" above):
- title slide makes main point
- 5-5-5 rule:
- 5 words per line
- 5 lines per slide
- <= 5 text slides in a row
- (details in notes)
- simple fonts, legible from back
- (more at www.ncsl.org/legislative-staff/lscc/tips-for-making-effective-powerpoint-presentations)
- data science report:
- ...
- executive one-pager
- ...
- We can seek help from Writing Center (I requested help Tu 6/16/26), e.g.:
- Academic and Professional Writing, writing.wisc.edu/handbook/assignments
- An Overview of the Writing Center for Instructors, writing.wisc.edu/instructor
- Collaborate with the Writing Center, writing.wisc.edu/instructor/outreach
- Writing Across the Curriculum (WAC), wac.wisc.edu
----------------------------------------------------------------------
- Week 8 Milestone: Formal 30-minute video conference with partners.
- Week 15 Milestone: Final on-campus presentations.
- Deliverable: Delivery of the Project Hand-off Package (Code,
Containers, Reports, and One-Pagers).
Guest lectures:
- Entrepreneurship: entrepreneurship.wisc.edu,
business.wisc.edu/entrepreneurship,
news.wisc.edu/lewis-sheats-named-inaugural-leader-of-uws-wisconsin-entrepreneurship-hub
- Heilmeier Catechism: www.darpa.mil/about/heilmeier-catechism