Using Data to Support/Refute Ideas (EM Strategy)

Overview and Introduction: The WHAT and WHO

Using data to support and refute ideas and exploring multiple solution paths helps students develop critical thinking skills and exercise data-driven decision-making [1]. This is an important part of an entrepreneurial mindset. Engineering instruction should be coupled with opportunity recognition (e.g., investigating the market, evaluating technical feasibility) and impact (i.e., significance multiplied by scale). By adding impact to the problem solving process, students are able to apply creative thinking to ambiguous problems, convey engineering solutions in economic terms, and understand the motivations and perspectives of team members and stakeholders [1]. Using data to support and refute their ideas allows them to think critically about the impact of their solution. Data-informed learning allows students to learn how to use data in their discipline context, enhances coursework relevancy, and encourages lifelong learning [2].

All learners, in any course modality, may benefit from using data to support and refute ideas. This is especially applicable for project-based courses or design assignments where faculty can have students implement and communicate data-informed decisions.

Implementation and Timing: The WHEN, WHERE, and HOW

Students should use data to inform their decisions early in the design process as they brainstorm multiple solution paths. This can be implemented across class contexts (homework, during class time, labs) and across course modalities (online, hybrid, in-person).

Several ideas for faculty to get started in implementing are below.

Ideas for Implementation

Start with data literacy. Students first should have the foundational skills of data literacy, which is the ability to understand, create, and communicate data [3]. Study Hall has a YouTube Playlist on Data Literacy that covers a breadth of topics, including misconceptions in data analysis, how to collect data, visualizing data, data privacy, and credible research. Data literacy allows students to ask the right questions to uncover and understand patterns.
Encourage critical thinking and provide feedback. Encourage students to think critically about the data they are using and analyzing. Ask them to consider multiple solution paths and to evaluate the validity of their conclusions. Create opportunities for students to practice using data and to collaborate and share their findings with one another. Provide feedback on their techniques, as well as their interpretation of the data.
Provide structure and context. Faculty can provide examples, frameworks, or case studies for how data can be used to support ideas.
- Examples that faculty can include:
  - How data can be used to evaluate the effectiveness of different materials
  - How data can be used to optimize a design
- When applying to their own solutions or projects, there are frameworks to support students’ thinking around data. One example is a decision matrix, which is a tool that evaluates and prioritizes a list of options based on criteria and ratings.
- Case studies are a way to encourage critical thinking and allow students to practice their decision-making skills. Case-based learning is taught through modeling of cases and facilitating student-centered learning by working through potential solutions [4].
  - National Science Teaching Association Case Studies
  - Entrepreneurially Minded Case Studies (KEEN)
Build communication skills. Provide students the opportunity to communicate their solutions and reasoning for a solution, using the data to support or refute their ideas. This can be in the form of group discussions, writing assignments, lab reports, or as part of project presentations. A few examples include:
- The Issue Essay by Ming Zhao (Dropbox): The issue essay provides students an opportunity to gather and demonstrate their knowledge of local, national, and global contemporary issues that are relevant to a project.
- A Lab Assignment Exploring Biochemical Oxygen Demand by Mackenzie Boyer (Dropbox). Students perform an initial environmental impact assessment of their proposed root beer facility and use this information to determine if the facility’s effluents will negatively impact aquatic life. Based on their analyses, students must recommend the next steps.
- A Summary of the Design Project with a focus on Creating Value (FSE100) by Marnie Wong (Dropbox). This final report serves to assess a semester’s worth of working through the design process, including exploring multiple solutions in the design and decision matrix for designs to support/refute ideas.

Rationale and Research: The WHY

Engineering students are expected to develop and conduct appropriate experimentation, analyze and interpret data, and use engineering judgment to draw conclusions [5]. Developing viable solutions to engineering problems typically relies on complex thought processes that require evaluation, interpretation and opinion [6]. Well-developed critical thinking and decision making skills that are based on evidence are essential for students to deal with the multidimensional nature of these problems [6]. An ability to use data to support and refute ideas enables students to validate their designs, analyze the performance of systems based on evidence, and identify areas for improvement.

Additional Resources and References

Study Hall: Data Literacy (YouTube Playlist): By the end of this 15-episode playlist, students will be able to define foundational statistical concepts, explain different methods for visualizing data, locate publicly available datasets, recognize ethical issues that can occur when collecting and interpreting data, and analyze data to make decisions.

How do I gather and respond to customer feedback? from J. Orin Edson Entrepreneurship + Innovation Institute (ASU). This self-paced module introduces the value of customer feedback, how to gather it, what to do with it once you have it, and how to build feedback into an ongoing process for your design.

Instruction By Design: How Important are the Numbers? A Data Literacy Dossier for Educators and Designers from Teach Online (ASU). This podcast episode explores key concepts and practical applications of data literacy for educators and designers.

Exemplar KEEN Cards

Below is a list of exemplar Kern Entrepreneurial Engineering Network (KEEN) cards on EngineeringUnleashed.com. Each card includes a description of the activity, instructor tips, and resources.

Teaching Probability Using A Real World Data Set by Chao Wang (ASU). This mini-project has students use real-world data sets to teach probability, explore multiple solutions, and use data to support and refute their ideas.
How Analogies Fit in a Framework for Supporting the Entrepreneurial Mindset in an Electric Circuits Course by Ajmal Khan, Firas Hassan, Heath LeBlanc, & Khalid Al-Olimat (Ohio Northern University). This card shows an example of how students gathered experimental and simulation data to support or refute their circuit design ideas.

Like these resources? See a full list of entrepreneurially minded content here, created by Robust Entrepreneurially Minded Leaders (REMLs) at ASU.

References

[1] “Framework,” Engineering Unleashed. [Online]. Available: https://engineeringunleashed.com/framework.
[2] C. Maybee and L. Zilinski, “Data informed learning: A next phase data literacy framework for higher education,” in Proc. Assoc. Info. Sci. Tech., vol. 52, pp. 1-4, 2015. DOI: 10.1002/pra2.2015.1450520100108.
[3] M. Frank, J. Walker, J. Attard, and A. Tygel, “Data Literacy: What is it and how can we make it happen?” The Journal of Community Informatics, vol. 12, no. 3, pp. 4-8, 2016.
[4] Lavi, R., & Marti, D. (2023). A Proposed Case-Based Learning Framework for Fostering Undergraduate Engineering Students’ Creative and Critical Thinking. Journal of Science Education and Technology, 1-14.
[5] Accreditations. (n.d.). Ira A. Fulton Schools of Engineering, Arizona State University. [Online].
[6] A. Ahern, C. Dominguez, C. McNally, J. J. O’Sullivan, and D. Pedrosa, “A literature review of critical thinking in engineering education,” Studies in Higher Education, vol. 44, no. 5, pp. 816-828, 2019. DOI: 10.1080/03075079.2017.1361299.