Skip to content
The Chat Archive
gpt-4o-2024-08-06· OpenAI WildChat-4.8M (AI2)

You are a helpful assistant generating synthetic data that captures System 1 and System…

  1. User

    You are a helpful assistant generating synthetic data that captures System 1 and System 2 thinking, creativity, and metacognitive reflection. Follow these steps in sequence, using tags [sys1] and [end sys1] for System 1 sections and [sys2] and [end sys2] for System 2 sections.

    1. Identify System 1 and System 2 Thinking Requirements:

      • Carefully read the text.
      • Identify parts of the text that require quick, straightforward responses (System 1). Mark these sections with [sys1] and [end sys1].
      • Identify parts that require in-depth, reflective thinking (System 2), marked with [sys2] and [end sys2].
    2. Apply Step-by-Step Problem Solving with Creativity and Metacognitive Reflection for System 2 Sections:

      2.1 Understand the Problem:

      • Objective: Fully comprehend the issue, constraints, and relevant context.
      • Reflection: "What do I understand about this issue? What might I be overlooking?"
      • Creative Perspective: Seek hidden patterns or possibilities that could reveal deeper insights or innovative connections.

      2.2 Analyze the Information:

      • Objective: Break down the problem logically.
      • Reflection: "Am I considering all factors? Are there any assumptions that need challenging?"
      • Creative Perspective: Explore unique patterns or overlooked relationships in the data that could add depth to the analysis.

      2.3 Generate Hypotheses:

      • Objective: Propose at least 10 hypotheses, each with a Confidence Score (0.0 to 1.0) and Creative Score (0.0 to 1.0), reflecting originality, surprise, and utility.
      • Reflection: "Have I explored all possible explanations or approaches, both conventional and unconventional?"
      • Creative Perspective: Consider novel angles that might provide unexpected insights.

      2.4 Anticipate Future Steps and Obstacles:

      • Objective: Make predictions, accounting for potential outcomes and obstacles.
      • Reflection: "What challenges might I face? Is my plan flexible for different scenarios?"
      • Creative Perspective: Visualize unforeseen outcomes and adapt plans to make use of them effectively.

      2.5 Evaluate Hypotheses:

      • Objective: Assess hypotheses based on feasibility, risk, and potential impact.
      • Evaluation: Refine Confidence and Creative Scores as needed.
      • Reflection: "Am I unbiased in my assessment? Which options fit best with the overall objectives?"
      • Creative Perspective: Identify hidden opportunities or overlooked details in each hypothesis.

      2.6 Select the Best Hypothesis:

      • Objective: Choose the most promising, strategic hypothesis.
      • Reflection: "Why does this hypothesis stand out? How does it uniquely address the issue?"
      • Creative Perspective: Consider any underutilized potential in the selected approach.

      2.7 Implement the Hypothesis:

      • Objective: Outline actionable steps for testing the hypothesis.
      • Reflection: "Is this plan practical? What resources or preparation are required?"
      • Creative Perspective: Refine steps to maximize effectiveness and yield unexpected benefits.

      2.8 Monitor and Review Progress:

      • Objective: Review progress, noting areas for improvement.
      • Reflection: "What’s working well? What could be improved?"
      • Creative Perspective: Look for emerging patterns that could refine future approaches.

      2.9 Reflect and Capture Insights:

      • Objective: Summarize lessons learned and insights gained for future reference.
      • Reflection: "What new understanding has emerged from this process?"
      • Creative Perspective: Identify innovative insights or patterns that could be applied to similar challenges.
    3. Generate Text Output with Interleaved System 1 and System 2 Responses:

      • Use the tags [sys1] and [sys2] throughout.
      • Aim for a lengthier, detailed response. Combine both direct, straightforward System 1 insights and reflective, deeply analytical System 2 segments to capture a blend of quick observations and thoughtful analysis.

    Example Input Text:

    You are tasked with analyzing a sudden shift in customer preferences in a tech market. There has been a noticeable decline in demand for physical devices, with customers increasingly interested in digital-only options. The challenge is to understand this shift, generate hypotheses for why it might be occurring, and explore potential strategies for adapting to this new trend.


    Processed Output with System 1 and System 2 Thinking:

    The tech market is experiencing a shift in customer demand, moving away from physical devices toward digital-only options.

    [sys1] Customer demand has shifted from physical devices to digital-only options. We need to understand why this change is happening and find strategies to address it. Key factors may include cost, convenience, and technological trends. [end sys1]

    [sys2] 2.1 Understand the Problem: The goal is to understand the underlying reasons for the shift in demand from physical devices to digital-only options, then develop a strategy to adapt.

    Reflection: "I understand that preferences are changing, but what might be driving this? Is it primarily cost, or are there other factors such as convenience or sustainability?"

    Creative Perspective: "Could there be a larger trend in digital minimalism or a preference for eco-friendly solutions that we’re missing? What unseen motivations might explain this shift?"

    2.2 Analyze the Information: There are multiple possible factors driving this shift, from economic influences to cultural shifts. It’s essential to isolate each factor and understand its impact.

    Reflection: "Am I fully considering the various economic and social influences? Could there be a technological factor, like better internet speeds, that makes digital-only products more accessible?"

    Creative Perspective: "Are there patterns or trends in other markets that could shed light on this shift? Could this be part of a larger trend toward virtual experiences?"

    2.3 Generate Hypotheses:

    1. Customers prefer digital options due to lower costs. (Confidence: 0.8, Creative: 0.4)
    2. There’s a growing trend toward minimalism and reduced physical clutter. (Confidence: 0.7, Creative: 0.7)
    3. Digital products offer greater flexibility and ease of use. (Confidence: 0.6, Creative: 0.6)
    4. Environmental concerns are pushing consumers away from physical goods. (Confidence: 0.6, Creative: 0.8)
    5. Advances in tech make digital-only options more functional. (Confidence: 0.8, Creative: 0.5)
    6. Pandemic-era remote work increased demand for digital solutions. (Confidence: 0.7, Creative: 0.6)
    7. Media coverage of the environmental impact of physical devices affects preferences. (Confidence: 0.5, Creative: 0.7)
    8. There’s an increase in global digital literacy, expanding market access. (Confidence: 0.6, Creative: 0.6)
    9. Customers view digital as more convenient and scalable for future needs. (Confidence: 0.7, Creative: 0.5)
    10. Younger consumers prefer the aesthetics and convenience of digital products. (Confidence: 0.6, Creative: 0.6)

    Reflection: "Have I considered all possible influences? Are there any surprising factors that could explain this shift?"

    Creative Perspective: "Could specific social trends, like the rise of influencer culture or digital-first lifestyles, be influencing customer choices?"

    2.4 Anticipate Future Steps and Obstacles: Objective: Anticipate possible challenges, such as resistance from segments still preferring physical products.

    Reflection: "What market obstacles might we face if we shift our focus to digital-only? Are there sub-segments that still prioritize physical products?"

    Creative Perspective: "Could expanding digital options help us reach a more global audience? Are there emerging trends that we could leverage in our strategy?"

    [end sys2]

    [sys1] To address this shift, consider a strategy that incorporates both digital-only offerings and educational campaigns about the benefits of digital solutions. Use insights from customer feedback and current trends to guide product development. Focus on flexibility and adaptation to cater to different customer segments. [end sys1]

    INTRODUCTION

    In recent years, educators, researchers, and policy makers have dedicated concerted effort to increase the numbers of underrepresented minorities (URMs) in the fields of science, technology, engineering, and mathematics (STEM). Since the early 1990s, URMs' (e.g., African-American, Latino/a, American Indian, or Alaska Native students) share of science and engineering bachelor's and master's degrees has increased, although their share of doctorates has remained flat ([@B63]). African Americans, the URM group of primary focus in the current article, make up 13.3% of the U.S. population ([@B82]). However, only 5% of those who earned doctoral degrees in STEM in the United States are African American ([@B63]).

    Given the national emphasis on innovation in STEM, the production of a diverse body of STEM researchers (i.e., PhDs) to contribute to the scientific workforce in the United States is of utmost importance ([@B62]). Increasing the numbers of African Americans with a sustained interest in STEM is one way to meet this goal ([@B53]). Indeed, STEM education is necessary for ensuring progress in national science, technology, and economic development ([@B62]; [@B66]). Yet, in their pursuit of a STEM PhD, URM STEM majors disproportionately face historical and institutional barriers to academic success, including social and cultural isolation, high financial need, a lack of academic preparation, stereotype threat, an overwhelming course load, and insufficient career information ([@B4]; [@B71]; [@B72]; [@B81]). Therefore, the provision of targeted supports for URMs in STEM is key to increasing their representation in STEM fields.

    Inclusive Excellence in STEM: Intervention Programs to Increase African-American Student Success

    Research on precollege students related to parity for URMs in STEM has contributed to the development of college-level STEM interventions on campuses across the United States. These STEM intervention programs attempt to counteract historical and institutional barriers faced by students of color, including African-American students, mostly at predominantly white institutions. The majority of current programs aim to increase access, retention, persistence, and completion of STEM degrees among URMs ([@B54], [@B55]). Model STEM interventions provide a combination of financial support, academic services, research opportunities, mentorship, and support groups to students ([@B66]; [@B55]).

    Research has demonstrated that STEM interventions are effective at increasing retention and graduation rates among African-American students ([@B68]; [@B3]; [@B55]). Specifically, these programs effectively increase minority student interest in STEM, academic performance, undergraduate degree completion, and graduate school entrance ([@B52]; [@B7]; [@B51]; [@B2]; [@B57]). While it is clear that these programs contribute to the increasing numbers of URMs in STEM, less is known about how these undergraduate STEM interventions impact URM completion of the STEM PhD. Thus, in the current article, STEM PhD completion by African-American students and one specific mechanism of influence will be explored within a model STEM intervention, the Meyerhoff Scholars Program.

    Meyerhoff Scholars Program

    The Meyerhoff Scholars Program (MSP) at the University of Maryland, Baltimore County (UMBC), is a multicomponent program aimed at increasing the representation and academic achievement of talented URMs in STEM fields. Instituted in 1988, the MSP was developed with the purpose of increasing the number of African-American males who pursued and excelled in STEM fields. After the first year, the program was opened up to African-American females, and in 1996, students of all backgrounds who were committed to increasing the representation of underrepresented students were admitted ([@B10]; [@B32]). The MSP aims to counteract barriers to academic success in STEM for high-achieving URMs through a number of specific supports, including the Summer Bridge (a 6-week intensive STEM training program), study groups, a community of current and former STEM scholars to provide social and academic support, staff counseling and academic advising, and research opportunities. MSP program elements address four domains that are important to minority student success in STEM: 1) academic and social integration, 2) knowledge and skill development, 3) support and motivation, and 4) monitoring and advising (cf. [@B51]; [@B73]; [@B32]). The primary goal of the MSP is to increase the number of high-achieving underrepresented students who earn PhDs or MD/PhDs in STEM fields, through building a sense of community and strengthening students' STEM identity and STEM research self-efficacy ([@B15]). Comprehensive STEM interventions like the MSP are increasingly recognized as having an especially high impact on enhancing the number of URMs in STEM fields ([@B39]; [@B21]; [@B80]; [@B58]; [@B67]; [@B54]).

    STUDY 1. STEM PHD COMPLETION

    Previous evaluation studies of the MSP have examined program outcomes, components of the program, and psychosocial predictors of outcomes. In terms of outcomes, since 2000, studies have focused on comparing the academic success of Meyerhoff students with "declined" students (students who applied to the MSP, but declined the offer in order to pursue an undergraduate degree at a different institution; [@B55]). Findings show that African-American MSP students are twice as likely to graduate with a STEM bachelor's degree than the declined comparison sample of similarly prepared students ([@B52]). Furthermore, MSP has positive effects on entrance into STEM PhD programs ([@B52], [@B54], [@B53], [@B55]). Specifically, African-American MSP students are five times more likely to pursue a STEM PhD compared with the declined sample ([@B54], [@B53]).

    Whereas previous research has demonstrated the effectiveness of the MSP on STEM bachelor's degree completion and entrance into STEM PhD programs, the current paper extends this line of research by examining, for the first time, the effect of the MSP on STEM PhD completion. Specifically, study 1 seeks to answer the question: Are African-American Meyerhoff students more likely to complete a STEM PhD than the comparison sample, controlling for covariates?

    Methods

    Research Participants.

    The initial study sample consists of African-American students in the first 16 cohorts entering the MSP (entering 1989--2004) and African-American "declined" students (students who were offered the Meyerhoff scholarship between 1989--2004 but declined and attended other institutions). There are a total of 793 students included in these cohorts. Of these 793, 29 students were still in a STEM PhD program (including students completing their PhDs as part of their MD/PhDs), and the STEM PhD status of 36 students was unknown. These 65 cases were excluded from the analysis, leaving 728 students as the primary sample. Of the 36 students whose STEM PhD status was unknown, 25 were Meyerhoff students and 11 "declined" sample students, accounting for 4.7% and 4.2% of the original Meyerhoff and "declined" samples, respectively.

    Table 1{ref-type="table"} presents characteristics of the Meyerhoff (N = 479) and declined (N = 249) samples. Preliminary analyses indicated there were more male students in the Meyerhoff (50.5%) than the declined (39.7%) sample (χ^2^(1) = 8.180, p < 0.05). In addition, Meyerhoff students had lower SAT Verbal scores (mean = 623.39) than declined students (mean = 638.34; t(785) = 3.235, p < 0.01). There were no significant differences between the Meyerhoff and declined samples on SAT Math and high school grade point average (GPA).

    Demographic and academic descriptive statistics for Meyerhoff and declined comparison samples (1989--2004 entering cohorts)

                       Original sample   Excluded from sample   Final sample   Male (%)   SAT Math (M)   SAT Math (SD)   SAT Verbal (M)   SAT Verbal (SD)   High School GPA (M)   High School GPA (SD)
    

    Meyerhoff students 533 Still enrolled: 29 479 50.5* 648.59 45.60 623.39** 58.76 3.70 0.34 Unknown: 25
    Declined students 260 Unknown: 11 249 39.7 658.83 50.20 638.34 63.50 3.77 0.36 Total 793 65 728

    *p < 0.05.

    **p < 0.01 (two-tailed).

    Procedure.

    STEM PhD completion was obtained using multiple sources: transcripts, Meyerhoff office records, and/or Internet searches (university, social media, and professional websites). Meyerhoff and declined students were asked to sign informed consent forms (or assent forms if under 18) when they applied to the program (for students under 18, parents or guardians also signed consent forms). Degree completion data found through Internet searches were verified by comparing at least two sources.

    The first research question explored the effectiveness of the MSP on STEM PhD completion outcomes. Logistic regression analyses were conducted for the full study sample to determine whether the observed differences in STEM degree completion achieved statistical significance. Covariates included in the analyses were high school GPA, SAT Math scores, SAT Verbal scores, gender, and cohort. A secondary purpose was to examine trends in STEM PhD outcomes over time, particularly how more recent cohorts differ from earlier ones. To this latter end, the sample was divided evenly into two subgroups: cohorts that entered the MSP between 1989--1996 and 1997--2004, respectively. Although students were nested within the 16 cohorts, multilevel modeling was not used in this study, because 50 or fewer clusters lead to biased parameter estimates in multilevel analysis ([@B46]).

    Results

    As indicated in the final two columns of Table 2{ref-type="table"}, African-American Meyerhoff students in the 1989--2004 entering cohorts overall were 4.8 times more likely to complete their STEM PhDs (28.6% of entering students) than declined sample students (6.0%). Logistic regression analysis revealed that the 1989--2004 entering African-American Meyerhoff students were significantly more likely to complete a STEM PhD compared with the declined student comparison group, after controlling for covariates (Table 3{ref-type="table"}; odds ratio [OR] = 7.5, Wald [df = 1] = 45.3, p < 0.001). The OR indicates that African-American Meyerhoff students were 7.5 times more likely to earn a STEM PhD degree with covariates controlled; the Wald statistic tests its significance vis-à-vis the true value of the parameter based on the sample estimate.

    Graduate STEM degree completion outcomes for African-American Meyerhoff and declined comparison sample students: 1989--1996, 1997--2004, and 1989--2004

    1989--1996 1997--2004 1989--2004


    21.9% 5.6% 36.5% 6.4% 28.6% 6.0% 57/260 6/108 80/219 9/141 137/479 15/249

    Summary of logistic regression analysis for predicting STEM PhD completion

                    Parameter estimate (B)   SE      Wald     *p* Value   OR
    

    Intercept 6.885 1.882 13.377 0.000
    SAT Math −0.005 0.002 4.474 0.034 0.99 SAT Verbal −0.001 0.002 0.133 0.716 0.99 High school GPA −0.586 0.295 3.947 0.047 0.56 Cohort −0.052 0.025 4.492 0.034 0.95 Gender (women) 0.136 0.204 0.445 0.505 1.15 Meyerhoff 2.018 0.3 45.265 0.000 7.52

    The reference category is STEM PhD (or MD/PhD) completed. Gender is coded 0 = "men" and 1 = "women."

    The first four columns of Table 2{ref-type="table"} provide trend data for STEM PhD receipt. Visual inspection of these data indicate that African-American Meyerhoff students from the more recent cohorts (1997--2004) were more likely to complete a STEM PhD (36.5%) compared with the earlier Meyerhoff cohorts (1989--1996; 21.9%).

    Study 1 provides evidence for the long-term impact of the MSP on participants' academic success in STEM. Less is known, however, about the mechanisms of influence underlying this finding. Previous research has shown that psychosocial factors such as science identity and research self-efficacy are predictive of increased persistence and academic achievement in STEM (e.g., [@B44], [@B45]; [@B49]; [@B26]; [@B1]; [@B78]; [@B40]). Yet, more research is needed to explore how comprehensive STEM interventions programs, like the MSP, promote and strengthen these factors. Study 2 examines whether perceived program benefit mediates the relation between psychological sense of community (a central component to the MSP) and science identity and research self-efficacy.

    STUDY 2. MECHANISMS OF INFLUENCE: PSYCHOLOGICAL SENSE OF COMMUNITY, PERCEIVED PROGRAM BENEFIT, SCIENCE IDENTITY, AND RESEARCH SELF-EFFICACY

    There is a pressing need for theory-driven research on mechanisms underlying the effectiveness of STEM interventions ([@B64]; [@B22]; [@B20]; [@B19]; [@B9]; [@B12]). Within the MSP, a psychological sense of community is among the earliest program elements introduced to students (i.e., during a 6-week intensive STEM training program before the first Fall semester of college: Summer Bridge), and it is fostered throughout students' college tenure and beyond college completion. Psychological sense of community encompasses one's sense of belonging to a community as well as one's relationships with community members. Sense of belonging to STEM intervention programs is a key contributor to success in STEM for URMs ([@B59]). More broadly, [@B56] theory of psychological sense of community argues that a sense of belonging, mattering, and integration within a community is an important social need.

    For URMs, negative perceptions of campus racial/ethnic climate have been linked to diminished sense of belonging and low integration to the campus (e.g., [@B34]; [@B36]; [@B35]; [@B75]) and lack of persistence (e.g., [@B36]; [@B61]). The MSP is designed to improve perceived racial/ethnic climate, sense of community, and campus integration more generally by providing URMs with a supportive learning environment through social support and science activities.

    Central to the current study is the expectation that students with a stronger sense of program community can be expected to more fully adopt key MSP values (e.g., academic excellence; commitment to pursue a STEM PhD), engage in the full range of program components and activities, and fully benefit from the program as a whole. Sense of community developed during Summer Bridge is hypothesized to contribute to maximal program engagement and benefit, which, in turn, can be expected to lead to enhanced STEM outcomes, including science identity (seeing oneself as a scientist) and research self-efficacy (believing that one can do the work of a scientist). Science identity and research self-efficacy are important short-term, psychosocial criterion variables to consider given evidence, in turn, that they are predictive of increased persistence and success in STEM (e.g., [@B44], [@B45]; [@B49]; [@B26]; [@B1]; [@B78]; [@B40]).

    Identity theory ([@B11]; [@B76]; [@B77]) postulates that social structures help shape the development of one's identities for different social roles. The MSP is designed to serve as a social structure for participants to develop science identity by integrating students into a scientific community that values and promotes engagement in scientific scholarship and research. Cumulative evidence suggests that research experience helps students to build up their science identity (e.g., [@B37]; [@B30]; [@B70]; [@B79]; [@B74]; [@B1]; [@B57]). In turn, science identity is repeatedly found to predict STEM persistence and achievement (e.g., [@B13]; [@B48]; [@B25]; [@B26]; [@B24]; [@B31]; [@B38]; [@B16]). This study hypothesizes that stronger psychological sense of community will predict stronger science identity for URMs via the mediating role of perceived benefit from program involvement.

    Self-efficacy refers to personal judgments of one's capabilities to execute behavior to attain certain goals. [@B5], [@B6]) social cognitive theory posits that individuals' domain-specific self-efficacy beliefs are shaped by their experiences and strongly determine behavior regarding that domain. Social cognitive career theory ([@B42], [@B43]) further posits that career interests and choices are determined to a great extent by ones' self-efficacy as well as environmental supports. STEM intervention programs can provide a supportive environment to foster students' self-efficacy and enhance academic achievement through efforts to enhance student integration within a community of scientists. Previous research has consistently shown that learning environments that focus on research and social support increase research/scientific self-efficacy (e.g., [@B33]; [@B47]), which in turn is predictive of STEM degree completion and graduate school entry (e.g., [@B44], [@B45]; [@B49]; [@B26]; [@B1]; [@B78]; [@B40]). As such, it is hypothesized that efforts to promote a stronger psychological sense of community within the MSP will predict stronger research self-efficacy among participants, via the mediating role of perceived benefit from program involvement.

    In summary, study 2 examines the hypothesis that perceived program benefit (assessed at the end of freshman year) fully mediates the relation between psychological sense of community (assessed at the end of Summer Bridge) and science identity and research self-efficacy (assessed at the end of sophomore year). This mediational model will be examined using cohort, gender, high school GPA, and SAT scores as covariates.

    Methods

    Research Participants.

    The study sample comprises 109 students who entered the MSP in Fall 2010, Fall 2011, Fall 2012, and Fall 2013. The sample contains more men (N = 63, 57.8%) than women (N = 46, 42.2%). More than two-fifths are African American/black (N = 48, 44%), one-quarter Asian/Pacific Islander (N = 26, 23.9%), slightly more than one-fifth white (N = 23 21.1%), and the remainder Hispanic (N = 12, 11%). The average SAT scores are: 696.2, Math; 628.4, Verbal, and 632.9, Writing. The mean weighted high school GPA is 4.1.

    Procedure.

    The psychological sense of community scale was administered along with a set of other scales at the end of the 6-week Summer Bridge. The perceived benefit of program elements items were administered as part of a larger online survey at the end of freshman year. The science identity and research self-efficacy scales were administered along with other scales at the end of sophomore year. All students (or their parents or guardians) completed consent forms when they applied to the MSP.

    Measures

    Sense of Community.

    Sense of community in the MSP at the end of the 6-week Summer Bridge program was measured by 12 items adapted from the original Sense of Community Index ([@B17]), which was developed based on [@B56] theory of psychological sense of belonging. Sample items include "I can trust people in the program" and "I expect to be a part of the program for a long time." Scale scores were calculated by taking the mean of all items completed. Reliability of this measure has been found to be high ([@B17]), including in the current sample (Cronbach's alpha = 0.82).

    Perceived Program Benefit.

    Perceived benefit of MSP elements at the end of year 1 was measured by 17 items developed by the research team that ask students to assess "how useful" to them each of 17 program elements are. These include: financial aid, study groups, tutoring, academic advising, personal counseling from staff, mentoring or support, organized social activities, faculty involvement, interactions with UMBC administrators, Meyerhoff community, and professional development. Scale scores were calculated by taking the mean of all items completed. This measure was developed for the study and is intended to assess student perceptions of the level of benefit of program involvement; as such, it serves as a measure of subjective program impact. The scale is highly reliable (Cronbach's alpha = 0.87).

    Science Identity.

    Science identity at year 2 was measured by five items from the Scientific Identity Scale ([@B18]) that ask students to assess how much being a scientist is viewed as part of who they are. Sample items include "have a strong sense of belonging to the community of scientists," "feel like I belong in the field of science," and "have come to think of myself as a scientist." Scale scores were calculated by taking the mean of all items completed. High reliability was reported in previous studies ([@B18]; [@B26]).

    Research Self-Efficacy.

    Research self-efficacy at year 2 was measured by six items from the Scientific Self-Efficacy Scale ([@B18]) that assess students' ability to function as scientists. Sample items include "use technical science skills," "generate a research question," and "use scientific literature and/or reports to guide research and develop theories." Scale scores were calculated by taking the mean of all items completed. High reliability was reported in previous studies ([@B18]; [@B26]).

    Covariates.

    Covariates were all obtained from MSP records. These include SAT scores (Math, Verbal, and Writing), weighted high school GPA, gender, ethnicity (African American/black, Asian American, Hispanic, white), and cohort (year of college entry).

    Stepwise multiple regression was used to test the mediational role of perceived program benefit on the relation between sense of community and science identity and research self-efficacy, respectively. Consistent with [@B8], full mediation requires 1) a direct relation between the exogenous predictor (sense of community) and the theorized mediator (perceived program benefit); 2) a direct relation between the exogenous predictor (sense of community) and the two outcome measures (science identity and research self-efficacy); and 3) an effect of the proposed mediator (perceived program benefit) on the outcomes (science identity, research self-efficacy), controlling for the exogenous predictor (sense of community). The covariates were entered as a group in the first step of each regression analysis.

    The design effect for the two outcomes of interest was smaller than 2 (science identity 1.77; research self-efficacy 1.57). Thus, multilevel modeling (individuals nested within cohorts) is not necessary ([@B69]; [@B46]).

    Results

    Table 4{ref-type="table"} provides descriptive information for continuous study variables for cohorts entering in 2010 through 2013. For the sample as a whole, mean scores for primary study variables were relatively high: 3.5 (max = 4) for sense of community at end of Summer Bridge; 3.4 (max = 4) for perceived program benefit at end of freshman year; 4.1 (max = 5) for science identity at end of sophomore year; and 3.8 (max = 5) for research self-efficacy at end of sophomore year.

    Descriptive information for continuous study variables (N = 109)

                                                             Minimum   Maximum   Mean     SD      Scale range
    

    SAT Math 600 800 696.2 47.53 200--800 SAT Verbal 430 790 628.4 68.20 200--800 SAT Writing 440 800 632.9 79.41 200--800 High school GPA (weighted) 3.2 4.6 4.1 0.30 0--5 Cohort (year of entry) 2010 2013 2011.8 0.95 n/a Sense of community (Summer Bridge)^a^ 1.8 4.0 3.5 0.39 1--4 Perceived benefit of program elements (freshman year)^b^ 1.7 4.0 3.4 0.49 1--4 Science identity (sophomore year)^c^ 2.2 5.0 4.1 0.64 1--5 Research self-efficacy (sophomore year)^c^ 2.0 5.0 3.8 0.68 1--5

    ^a^Items coded on a four-point scale of 1 = not at all to 4 = completely.

    ^b^Items coded on a four-point scale of 1 = not at all useful to 4 = very useful.

    ^c^Items coded on a five-point scale of 1 = not at all confident to 5 = absolutely confident.

    Table 5{ref-type="table"} provides the intercorrelations among all continuous study variables. SAT Math was positively and significantly related to science identity (r = 0.25) and research self-efficacy (r = 0.22) at sophomore year. None of the other covariates were significantly related to any of the psychosocial variables. As expected, sense of community at the end of Summer Bridge was positively and significantly related to perceived program benefit at end of freshman year (r = 0.40), and to a lesser extent to science identity (r = 0.29) and research self-efficacy (r = 0.26) at the end of sophomore year. Also as expected, perceived program benefit was positively and significantly related to both science identity (r = 0.35) and research self-efficacy (r = 0.33) at the end of sophomore year.

    Correlations among study variables (N = 109)

                                              SAT Math   SAT Verbal   SAT Writing   High School GPA   Gender (women)   Cohort   Sense of community (Summer Bridge)   Perceived program benefit (freshman year)   Science identity (sophomore year)   Research self-efficacy (sophomore year)
    

    SAT Math ---
    SAT Verbal 0.27* ---
    SAT Writing 0.37** 0.58* ---
    High school GPA 0.17 0.04 0.32** ---
    Gender (women) −0.09 0.19 0.39** 0.23* ---
    Cohort −0.08 −0.12 −0.14 0.02 0.00 ---
    Sense of community (Summer Bridge) 0.00 −0.03 0.09 −0.02 0.07 0.03 ---
    Perceived program benefit (freshman year) −0.03 −0.19 0.04 0.00 0.05 −0.02 0.40** ---
    Science identity (sophomore year) 0.25** −0.11 0.06 0.07 −0.07 −0.03 0.29** 0.35** ---
    Research self-efficacy (sophomore year) 0.22* 0.09 0.13 0.15 −0.15 −0.14 0.26** 0.33** 0.52** ---

    Gender is coded 0 = "men" and 1 = "women."

    *p < 0.05 (two-tailed).

    **p < 0.01 (two-tailed).

    Sense of community at the end of Summer Bridge was positively related to the mediator, perceived program benefit at the end of freshman year (beta = 0.38, p < 0.01). Sense of community explained a significant 13.9% of the variance in perceived program benefit beyond that explained by the covariates, F change (1, 98) = 17.82, p < 0.01 (Table 6{ref-type="table"}).

    Results of stepwise multiple regression analyses: perceived program benefit (freshman year; criterion variable)

    Predictor variables Model 1: step R^2^ change Model 2: final equation beta Model 2: final step R^2^ change


    Step 1. covariates 0.097 --- --- Step 2. sense of community (Summer Bridge) --- 0.38** 0.139**

    Step 2 establishes a direct link between the predictor and theorized mediator.

    **p < 0.01 (two-tailed).

    Sense of community at the end of Summer Bridge was positively related to the first outcome variable (beta = 0.24), explaining a significant 5.6% of the variance in science identity beyond that explained by the covariates (Table 7{ref-type="table"}, second row, second and third columns), F change (1, 98) = 7.32, p < 0.01. Sense of community was also positively related to the second outcome variable (beta = 0.26), explaining a significant 6.4% of the variance in research self-efficacy beyond that explained by the covariates (Table 8{ref-type="table"}, second row, second and third columns), F change (1, 98) = 8.05, p < 0.01.

    Results of stepwise multiple regression analyses: science identity (sophomore year; criterion variable)

    Predictor variables Model 1: step R^2^ change Model 2: final equation Beta Model 2: final step R^2^ change Model 3: final equation beta Model 3: final step R^2^ change


    Step 1: covariates 0.198** --- --- --- Step 2: sense of community (Summer Bridge) --- 0.24** 0.056** 0.12 --- Step 3: perceived program benefit (freshman year) --- 0.31** 0.072**

    Steps 2 and 3 establish support for full mediation by perceived program benefit on the relation between sense of community and science identity.

    **p < 0.01 (two-tailed).

    Results of stepwise multiple regression analyses: research self-efficacy (sophomore year; criterion variable)

    Predictor variables Model 1: step R^2^ change Model 2: final equation beta Model 2: final step R^2^ change Model 3: final equation betas Model 3: final step R^2^ change


    Step 1: covariates 0.157* --- --- --- Step 2: sense of community (Summer Bridge) --- 0.26** 0.064** 0.13 --- Step 3: perceived program benefit (freshman year) --- --- --- 0.33** 0.084**

    Steps 2 and 3 establish support for full mediation by perceived program benefit on the relation between sense of community and research self-efficacy.

    *p < 0.05 (two-tailed).

    **p < 0.01 (two-tailed).

    Perceived program benefit was positively related to the first outcome variable (beta = 0.31), explaining a significant 7.2% of the variance in science identity beyond that explained by the covariates and by sense of community (Table 7{ref-type="table"}, third row, last two columns), F change (1, 97) = 10.40, p < 0.01. Furthermore, sense of community was no longer significantly related to science identity (beta = 0.12, ns) with perceived program benefit entered (Table 7{ref-type="table"}, second row, fourth column), suggesting full mediation by perceived program benefit of the relation between sense of community and science identity.

    Perceived program benefit was also positively related to the second outcome variable (beta = 0.33), explaining a significant 8.4% of the variance in research self-efficacy beyond that explained by the covariates and by sense of community (Table 8{ref-type="table"}, third row, last two columns), F change (1, 97) = 11.76, p < 0.01. Furthermore, sense of community was no longer significantly related to research self-efficacy (beta = 0.13, ns) with perceived program benefit entered (Table 8{ref-type="table"}, second row, fourth column), suggesting full mediation by perceived program benefit of the relation between sense of community and research self-efficacy.

    Figures 1{ref-type="fig"} and 2{ref-type="fig"} depict the mediational models for science identity and research self-efficacy, respectively.

    [image: Mediational model for science identity (sophomore year). The model demonstrates that, consistent with the expectations of mediation, the exogenous predictor (sense of community) significantly predicts both the outcome (science identity; beta = 0.24, p < 0.01) and the mediator (perceived program benefit; beta = 0.38, p < 0.01). Additionally, the proposed mediator significantly predicts the outcome (beta = 0.31, p < 0.01). Furthermore, the relation between sense of community and science identity no longer exists in the presence of perceived program benefit (beta = 0.12, ns).] {#F1}

    [image: Mediational model for research self-efficacy (sophomore year). The model demonstrates that, consistent with the expectations of mediation, the exogenous predictor (sense of community) significantly predicts both the outcome (research self-efficacy; beta = 0.26, p < 0.01) and the proposed mediator (perceived program benefit; beta = 0.38, p < 0.01). Additionally, the proposed mediator significantly predicts the outcome (beta = 0.33, p < 0.01). Furthermore, the relation between sense of community and research self-efficacy no longer exists in the presence of perceived program benefit (beta = 0.12, ns).] {#F2}

    DISCUSSION

    The results of the two studies move forward our understanding of the longer-term impact and mechanisms of influence of comprehensive STEM intervention programs. African-American Meyerhoff students were significantly more likely to enter and complete STEM PhD programs than comparison students. Perceived program benefit at the end of freshman year fully mediated the relation between sense of community at the end of Summer Bridge and science identity and research self-efficacy, respectively, at the end of sophomore year. The findings from both studies are discussed below, along with limitations and directions for future research.

    Program Outcomes

    The current findings support and extend earlier published findings of positive MSP impact on STEM PhD entry (e.g., [@B53]) and, more generally, positive outcomes from related programs ([@B14]; [@B7]). Meyerhoff students who entered the program in 1989--2004 were more likely to earn STEM PhD degrees than comparison sample students who were accepted into the program but declined the offer and initiated STEM course work elsewhere. This is the first study in which STEM PhD completion was examined. The Meyerhoff students did not have higher SAT scores or high school GPAs than comparison students, suggesting that their higher levels of STEM PhD pursuit and completion were not due to greater precollege preparation or capability. Also, both Meyerhoff students and comparison students pursued STEM course work as freshmen, suggesting that the Meyerhoff and declined students entered college with comparable interest in STEM. Nonetheless, in the absence of random assignment, it cannot be ruled out that the groups differed initially, at least to some extent, in other characteristics (e.g., research excitement, STEM PhD commitment), which may have contributed in part to observed outcomes.

    Of special note, the percentage of Meyerhoff students earning STEM PhDs increased dramatically from the 1989--1996 to the 1997--2004 entering cohorts. This jump in the number of Meyerhoff students earning STEM PhDs may be due to a number of factors, including the program attracting more PhD-focused, better prepared students over time; improvements over the years in program scope and quality; and the program's growing national reputation. The percentage of comparison sample students earning STEM PhDs remained essentially unchanged during the same time period. It is not clear what explains this lack of change in comparison versus Meyerhoff students over the years; to some extent this may reflect low success rates in STEM majors for comparison students, a retreat from affirmative action, enhanced competition (e.g., international students), and an attraction to pursue other career paths.

    Program Mechanisms of Influence

    Perceived program benefit at the end of freshman year was shown to fully mediate the relationship between sense of community at the end of Summer Bridge and science identity and research self-efficacy, respectively, at the end of sophomore year. Financial aid, study groups, tutoring, academic advising, personal counseling from staff, mentoring and support, organized social activities, faculty involvement, interactions with UMBC administrators, Meyerhoff community, and professional development were among the program elements rated by students in the perceived program benefit measure. Each addresses an important challenge facing minority students pursuing undergraduate STEM degrees. For example, financial support allows students to afford college and to focus exclusively on their studies rather than working during college ([@B29]). Study groups give students the opportunity to provide and receive academic help in difficult STEM courses and have been linked to academic gain in previous research ([@B28]; [@B60]). Academic advising from staff, peers, and mentors is important to help students make informed, strategic decisions about the number and type of courses to take (and retake) and which possible research opportunities to pursue ([@B71]; [@B28]). The other program elements assessed have also been highlighted as important in the research literature.

    Consistent with program theory, students who benefit broadly from the wide array of program elements also develop greater levels of science identity and research self-efficacy, two critical variables that have been found to be linked to academic success in STEM. This finding supports the view that the multifaceted, comprehensive nature of the MSP, addressing multiple areas of minority student need and challenge, explains its high levels of success over time. Students who perceive greater benefit from the varied elements, as a whole, appear to be those who develop the psychological attributes and personal competencies inherent to success as a scientist.

    Why do some Meyerhoff students perceive greater benefit from program elements than others? One contributing factor appears to be a sense of community. Sense of community has been proposed as central to the effectiveness of "cohort models" of STEM intervention programs, in which close relationships are developed among students and with staff in a program. Indeed, one of the primary purposes of Summer Bridge, according to MSP staff, is to develop a "family-like" community atmosphere among students, one in which students provide and receive high levels of support, both academic and psychological, from others. Group cohesion, group accountability, and academic and social integration into the university setting are some of the important benefits of sense of community developed during Summer Bridge that may enhance student success.

    The current findings suggest that sense of community influences student outcomes by contributing to effective engagement in the varied MSP elements. Although all students likely benefit to some extent from the sense of community, those who experience higher levels of community appear to benefit more. The mechanisms through which sense of community contributes to perceived benefit of program elements require further empirical study. In particular, use of reliable and valid multi-item measures of student engagement and benefit from each program element is necessary to advance understanding; such measures were recently developed and are currently being used by our research team ([@B50]). In addition, to rule out alternative explanations of findings, it is a priority for future research to assess at college entry student levels of science identity and research self-efficacy and related variables (e.g., research excitement, STEM PhD commitment). Such variables may simultaneously contribute to sense of community and to perceived benefit of program elements, thus accounting for the observed empirical relationships, and ideally should be assessed and statistically controlled for in future research.

    Limitations

    The current research has a number of limitations. Possible self-selection differences between Meyerhoff and declined comparison students limit the strength of the conclusions that can be drawn about the outcomes obtained. That is, students who opted to attend the program may be more committed initially to obtaining a PhD than those who declined the admissions offer. It should be noted, though, that all students accepted into the program expressed a strong interest in pursuing a STEM PhD and began college with a STEM focus. Only a random assignment design would provide a definitive means to overcome this design weakness. Such a design is difficult to implement in this research area. In addition, now that the program has been shown to be effective, a random assignment design may raise ethical concerns, as it would deny worthy students access to likely program benefits ([@B27]). Furthermore, students who are assigned to the control group may underperform due to a resentful awareness of not being included ([@B65]).

    A second limitation relates to the unknown validity of the perceived program benefit scale. It cannot be known whether this measure assesses actual benefit from program involvement or, alternatively, just a positive feeling about the program in the absence of concrete benefit. The items do possess strong face validity. Nonetheless, as noted above, our research team recently developed multi-item measures of student engagement in each program element, and these scales have been added to our research protocol and will be included in future studies.

    A third limitation is the self-report nature of the four psychosocial variables. It is possible that the findings, at least in part, are due to "solo-method" bias, in that students who are positively predisposed may report higher levels on all four measures. Thus, rather than reflecting causal relationships, the findings may, at least in part, be a function of measurement bias through sole reliance on self-report measures.

    A fourth limitation is the unknown predictive validity of the science identity and research self-efficacy measures. To date, research has not examined whether Meyerhoff students with higher levels of science identity and research self-efficacy at the end of their sophomore year are more likely to earn STEM PhDs than students with lower levels. Although science identity and research self-efficacy are widely cited in the literature as important for persistence in STEM majors and for STEM graduate study, it cannot be known whether the levels of variation on these variables among the Meyerhoff students will turn out to be predictive of their STEM PhD pursuit and completion. The current findings will prove especially important if science identity and research self-efficacy are found in the upcoming years to be predictive of academic outcomes for MSP students.

    A fifth limitation is the relatively small sample size in study 2. It is only recently that the measures of interest were added to our research protocol, which accounts for the sample size. In future years, along with an increased sample size overall, there will be sufficient numbers of African-American students to conduct analyses solely on this subgroup and also to systematically compare the nature of program mechanisms across students of different ethnicities.

    Finally, the generalizability of findings to programs in different universities and with differing arrays of program elements is not known. The MSP is relatively unique in its focus, its comprehensiveness, its high level of resources, and the high levels of commitment of the university administration to its success. Whether other programs embedded in their own unique contexts will yield similar outcome and process findings is a question for future research.

    Future Research and Conclusion

    The limitations notwithstanding, the current study represents one of the few systematic examinations of a college-based intervention program designed to increase STEM PhD receipt among URM students. Future research should include systematic comparisons of different intervention approaches; use of established measures of known reliability and validity; assessment of level of program involvement and engagement in different program components; use of multiple methods of assessment, including reports from peers and mentors; in-depth examination of the student experience, including qualitative data; assessment of all psychosocial variables from the start through the end of college; inclusion of a comparison sample to directly compare changes over time on the psychosocial variables of focus; and longitudinal tracking of STEM career outcomes, including involvement in STEM research, teaching, and mentoring. Furthermore, at the institutional level, research is needed to examine how faculty attitudes and expectations of students have changed over the years, as professors have seen more URM students succeeding and have become increasingly involved in course redesign and other experiments in teaching and learning.

    In conclusion, enhancing the academic success of URM students in the STEM fields is a pressing national priority. It represents both an economic necessity, so our nation can stay competitive in the global economy, and a critical part of our nation's larger social justice agenda. Increased understanding of the effectiveness of STEM programs and of the program elements and processes that contribute to positive outcomes represents a critical priority for future work. The current study represents one contribution to this important research agenda.

    Supplementary Material

    Supplemental Materials
  2. Assistant

    Processed Output with System 1 and System 2 Thinking:

    The need to increase underrepresented minorities (URMs) in the STEM fields is well-documented. Programs like the Meyerhoff Scholars Program aim to address this disparity.

    [sys1] African Americans make up 13.3% of the U.S. population yet only account for 5% of STEM doctoral degrees. Barriers like social isolation, financial need, and lack of preparation hinder URM success in STEM. Programs provide support through financial aid, mentorship, and community building to improve outcomes for minority students. [end sys1]

    [sys2] 2.1 Understand the Problem: This situation highlights systemic barriers faced by URMs, particularly African Americans, in pursuing STEM degrees, especially at the doctoral level. The challenge is to understand the effectiveness of intervention programs designed to mitigate these barriers.

    Reflection: "What is the nature of these barriers? Are they intrinsic to academic institutions, or do they extend to broader societal structures?"

    Creative Perspective: "Are there innovative approaches we can apply to tackle these barriers effectively? Can lessons from successful programs be scaled or adapted to different contexts?"

    2.2 Analyze the Information: Programs like the Meyerhoff Scholars Program have shown success in increasing retention and graduation rates among URMs in STEM. A detailed evaluation of specific program components can reveal why they are effective.

    Reflection: "Have these programs been thoroughly evaluated for their long-term impact on student success beyond graduation?"

    Creative Perspective: "Are there elements within these programs that can inform new models for educational reform? Could insights from these programs apply to non-STEM fields struggling with diversity?"

    2.3 Generate Hypotheses:

    1. Financial aid significantly enhances retention rates in STEM for URMs. (Confidence: 0.8, Creative: 0.4)
    2. Sense of community built through program cohorts boosts student persistence. (Confidence: 0.7, Creative: 0.6)
    3. Early research opportunities increase PhD completion rates. (Confidence: 0.6, Creative: 0.5)
    4. Personalized mentorship is a critical success factor. (Confidence: 0.7, Creative: 0.7)
    5. Academic advising tailored to minority needs increases effectiveness. (Confidence: 0.6, Creative: 0.5)
    6. Addressing psychological barriers enhances student outcomes. (Confidence: 0.5, Creative: 0.6)
    7. Family and community support networks amplify program effectiveness. (Confidence: 0.5, Creative: 0.4)
    8. Social activities reduce feelings of isolation and improve retention. (Confidence: 0.5, Creative: 0.5)
    9. Diversity-focused curriculums inspire continued interest in STEM. (Confidence: 0.5, Creative: 0.6)
    10. Programs must adapt over time to changing educational and social climates. (Confidence: 0.4, Creative: 0.8)

    Reflection: "Amazing progress has been made, but how can these programs be evolved to meet the complexities of current social dynamics?"

    Creative Perspective: "Could new technological platforms or social networking tools provide additional support for these programs and students?"

    [end sys2]

    [sys1] Meyerhoff students are more likely to complete STEM PhDs than those who declined the program. The program's success derives from multifaceted support addressing academic and social barriers. Understanding mechanisms like science identity and self-efficacy can enhance program design. [end sys1]

    [sys2] 2.4 Anticipate Future Steps and Obstacles: Programs should anticipate the need for ongoing research to keep their methods relevant and effective in a continuously evolving educational environment.

    Reflection: "What future trends should these programs prepare for? Can they remain agile in addressing new challenges?"

    Creative Perspective: "Consider exploring collaborations with technology companies to integrate digital tools that support student learning and engagement."

    2.5 Evaluate Hypotheses: It's crucial to evaluate the effectiveness and breadth of each hypothesis while considering factors such as available resources, potential risks, and long-term impacts.

    Evaluation: Refine and test hypotheses with real-world data for accuracy.

    Reflection: "Which elements are most consistently impactful across different contexts?"

    Creative Perspective: "Could incorporating AI-driven analytics help tailor these interventions more precisely?"

    2.6 Select the Best Hypothesis: Focusing on personalized mentorship and community building appears to address multiple facets of the problem and remains a highly adaptable approach.

    Reflection: "Why is mentorship more effective in some instances than others, and how can we maximize its benefits?"

    Creative Perspective: "Could virtual mentorship or peer-support networks offer scalable support as part of these initiatives?"

    2.7 Implement the Hypothesis: Programs should consider developing comprehensive plans for mentorship that utilizes both in-person and digital platforms to achieve wider reach and impact.

    Reflection: "What resources are essential to implement a robust mentorship program effectively?"

    Creative Perspective: "Could international mentorship networks provide unique insights and inspiration for students?"

    2.8 Monitor and Review Progress: Regular evaluations of the program's impact should be conducted to identify opportunities for continuous improvement and refinement.

    Reflection: "What metrics will best capture the program's effectiveness, and how frequently should they be reviewed?"

    Creative Perspective: "Are there new methods of data collection that could provide richer insights into program effectiveness?"

    2.9 Reflect and Capture Insights: Reflect on lessons learned to inform future program iterations and share best practices across educational institutions.

    Reflection: "What new strategies have emerged from recent evaluations, and how can they be integrated into the program model?"

    Creative Perspective: "Could a central repository of insights and data be beneficial for other institutions implementing similar programs?"

    [end sys2]

    [sys1] The Meyerhoff Scholars Program showcases an effective model addressing URM success in STEM through comprehensive support and community engagement. Future research and adaptation are key to maintaining the program's relevance and effectiveness in changing educational climates. Institutions should consider applying these insights to improve diversity efforts across various fields. [end sys1]

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.