Homework 3 ✍🏻

Back to the Data Engineering Course - 2024 Spring (AI5308/AI4005)

Homework 3 is due on Sunday, May 19, at 11:59 PM.

This assignment comprises a paper critique and a design problem with a programming component. Completing this assignment will provide practical experience in designing and implementing modules that address critical issues in AI systems, deepening your understanding of how continual learning and responsible AI principles can be applied in real-world applications.

  • Deliverables:
    • Format: Submit a PDF of your homework to the Gradescope under “HW3”. You may use LaTeX, Word, or submit neatly handwritten and scanned solutions.
    • Organization: Start each question on a new page and include any graphs within the relevant sections. Each solution should be self-contained on its own page.
    • Collaboration: List the names of students who assisted you, or those you assisted, with the homework. Exchanging code is not allowed.
    • Gradescope Submission: Match each page to the corresponding question when submitting your homework. See this video for Gradescope tutorial.
  • Contents:
    1. Paper Critique: Choose one of the following topics and read the designated chapters from the DMLS book along with relevant materials. Write a critique in accordance with these guidelines. You have two choices:
    2. Design and Programming: Integrating Advanced Modules for Data Shift and Responsible AI
      • Objective: Address data shift and continual learning or responsible AI issues in machine learning models. Apply these concepts to your team project or an individual model you developed in HW2.
      • Requirements: Submit a document (up to 4 pages) covering:
        1. Issue Identification:
          • Topic I: Describe expected data shift or continual learning issues.
          • Topic II: Identify potential ethical concerns (e.g., bias, fairness) in your model.
        2. Module Design: Design a module tailored to mitigate the identified issue in your model. Explain the basis of your module and how it integrates with the overall system architecture.
        3. Model Development: Detail the development and integration of the module with the main model (or pipeline).
        4. Results and Analysis:
          • Discuss the outcomes of integrating the module. Include an evaluation of its effectiveness.
          • Utilize Weights & Biases to monitor and visualize the model’s performance related to Topic I or II. Provide supporting graphs and visual aids. (Reference material by Sungho (TA))
      • Additional Resource: For more insights and practical lab exercises on building AI modules, check out the MIT Introduction to Data-Centric AI: Data-Centric AI Labs.
  • Grading Criteria:
    • Paper Critique (5%):
      • Check Plus (5%) - The critique is very well written and very insightful.
      • Check (3-4%) - Adequate. Most critiques are expected to fall into this category.
      • Check Minus (2%) - The critique lacks depth. Summaries may be vague, strengths/weaknesses trivial, questions superficial, and discussions shallow.
      • No submission (0%)
    • Design and Programming (10%):
      • Exceptional (9-10%): Innovative module design with seamless integration. Comprehensive and insightful analysis with exemplary use of Weights & Biases. Visual aids significantly enhance understanding.
      • Adequate (7-8%): Competent module design and integration. Solid analysis with good use of Weights & Biases. Visual aids are supportive.
      • Needs Improvement (5-6%): Basic module design with integration issues. Limited analysis and minimal use of Weights & Biases. Visual aids lack clarity.
      • Insufficient (3-4%): Poor module design and integration. Inadequate analysis and insufficient use of Weights & Biases. Visual aids are incorrect or irrelevant.
      • No submission (0%)
    • Late submission will be graded according to the late policy