Standard 4. Creating an Organization Committed to Quality Assurance, Institutional Learning, and Improvement
The institution engages in sustained, evidence-based, and participatory self-reflection about how effectively it is accomplishing its purposes and achieving its educational objectives. The institution considers the changing environment of higher education in envisioning its future. These activities inform both institutional planning and systematic evaluations of educational effectiveness. The results of institutional inquiry, research, and data collection are used to establish priorities, to plan, and to improve quality and effectiveness.
CFR 4.1
The institution employs a deliberate set of quality-assurance processes in both academic and non-academic areas, including new curriculum and program approval processes, periodic program review, assessment of student learning, and other forms of ongoing evaluation. These processes include: collecting, analyzing, and interpreting data; tracking learning results over time; using comparative data from external sources; and improving structures, services, processes, curricula, pedagogy, and learning results.
Stanford’s quality assurance for degree programs is accomplished through program review. New degree programs go through a faculty-governed process run by an appropriate committee of the faculty senate. As described under CFR 2.7, C-RUM and C-USP have oversight of the undergraduate curriculum, and C-GS oversees graduate degree programs. C-RUM reviews new undergraduate degree programs, interdisciplinary degree programs (IDPs), and is instituting a new process to review issues of universitywide concern for all majors (see Theme 2 of our report). Each of the schools offering undergraduate degrees reviews departmental major programs every five to eight years. A number of faculty boards including the FYGB and the BGB oversee various aspects of the undergraduate general education curriculum (see CFR 2.2). In addition to the review processes for academic degrees, external and internal reviews are conducted into many of our extracurricular and student support programs, such as those programs housed under the Vice Provost for Student Affairs (VPSA) and the Vice Provost for Undergraduate Education (VPUE). For example, in 2017-8, an external review of professional advising led to an increase in the cadre of academic advisors and made other recommendations about strengthening the ties between professional and major advising. Continuation of this review is ongoing with further study about emerging needs post-pandemic.
Stanford has made significant investments in building an infrastructure to support assessment of both curricular and extracurricular programs. Institutional Research & Decision Support (IR&DS) has developed a suite of student surveys that enable longitudinal and trend analysis and are leveraged extensively for academic and non-academic program review by units across the university. The strategy of having a core suite of surveys administered centrally reduces the need for individual units to conduct their own surveys, increasing the reliability and quality of the data and reducing the survey burden on students. Findings from IR&DS surveys are available here. IR&DS participates in peer consortia to collect and analyze comparative surveys and student success data so that university leaders understand Stanford’s results in the context of broad trends among highly selective institutions.
In addition to resources provided centrally by IR&DS, administrative units across the university have invested in analytical teams who work closely with campus stakeholders to conduct local assessments and program evaluations. An example is the Assessment and Program Evaluation team within the VPUE, which works closely with VPUE programs to: clarify and articulate measurable program goals and objectives; help programs articulate a theory of action or logic model for their work; and design and carry out program evaluation studies. The work of this team is cited extensively in Theme 1 of our institutional report to WSCUC and in many of the appendices to our report.
To support the assessment of program learning outcomes, IR&DS maintains a website with assessment resources and templates. IR&DS staff with expertise in learning outcomes assessment are available to consult with departments as they design their program learning outcomes assessment efforts.
CFR 4.2
The institution has institutional research capacity consistent with its purposes and characteristics. Data are disseminated internally and externally in a timely manner, and analyzed, interpreted, and incorporated in institutional review, planning, and decision-making. Periodic reviews are conducted to ensure the effectiveness of the institutional research function and the suitability and usefulness of the data generated.
In 2008, as Stanford began the preparations for self-study and reaccreditation, university leaders created a central office to consolidate and strengthen Stanford’s institutional research capacity. This new office of Institutional Research & Decision Support (IR&DS) brought together data analysts from functional areas, such as the registrar’s office and admissions, and partnered them with experts in data warehousing and business intelligence reporting. Since then, Stanford has continued its investment in building institutional research capacity both in IR&DS and in schools and administrative units. The IR&DS staff has grown from 10 members at founding to over twenty staff members in 2022, organized into four teams, each with their own mission and methods (see IR&DS organizational chart). The IR&DS teams focus on data governance, assessment and evaluation, institutional research, and data warehousing/business intelligence.
Stanford’s IR&DS is unusual among peer offices in combining the IR function with business intelligence reporting. Through the Student Integrated Reporting and Information System (SIRIS), IR&DS provides reporting to front-line campus staff on student, course, instruction, and advising data. The data management and analytic capabilities of the Decision Support Team within IR&DS were particularly critical during the early months of the COVID pandemic as departments rapidly shifted their curriculum to remote instruction. IR&DS quickly built a suite of new dashboards highlighting changes in course enrollment and enrolled student characteristics to support departments in curriculum planning.
The mission of IR&DS has changed substantially over the years. At the time of the office’s founding, the core purpose was to provide university leadership with information for decision support. As the university has developed a strong culture of evidence and appetite for data, the audiences of the IR&DS organization have grown to include faculty and staff across the institution, students, and the public. IR&DS does extensive work for faculty senate committees, supporting Stanford’s tradition of faculty governance. IR&DS has embraced its role in supporting increased institutional transparency by publishing public reports and dashboards, many of which are found on the Data and Findings page of the IR&DS website.
The IR&DS office has undergone periodic review to ensure the effectiveness of the function. An external review of the office was undertaken in 2010, led by institutional research personnel from peer institutions. To ensure that university leaders were receiving the information they needed for decision-making and strategic initiatives, in 2013, the provost charged an IR&DS Advisory Committee composed of school deans and vice provosts. This committee meets quarterly and provides input into IR&DS priorities, feedback on IR&DS products, and makes policy recommendations regarding the governance of sensitive data.
In addition to the central IR&DS office, many schools and administrative units have internal teams performing reporting, analysis, and evaluation functions. These distributed analytical teams are well-placed to understand the context and needs of their schools and organizations. There can, however, be challenges with duplication of effort and conflicting findings. Closer coordination and communication among the analytical teams serving the university would be beneficial
CFR 4.3
Leadership at all levels, including faculty, staff, and administration, is committed to improvement based on the results of inquiry, evidence, and evaluation. Assessment of teaching, learning, and the campus environment—in support of academic and co-curricular objectives—is undertaken, used for improvement, and incorporated into institutional planning processes.
Stanford’s leadership demonstrates a strong commitment to using evidence and evaluation to drive institutional change. A clear example of the university’s commitment to using evidence in support of institutional improvement was the first-ever community wide DEI survey, conducted in May, 2021, which is described at length in Theme 2 of our self-study. University leadership has also committed to making consistent demographic data about the university community available publicly through the IDEAL dashboards, building the university’s culture of evidence and accountability.
Broadly, the campus community has developed a strong culture of leveraging inquiry and evidence in support of decisions and governance processes. Findings from surveys and institutional data are routinely presented in the faculty senate, committees, and task forces across the university. As examples of the use of data collected from the community to drive campus improvements, student and parent surveys have informed changes to the cost of attendance, enhancements to our financial aid program, and policy changes prohibiting the use of course-specific fees. Surveys of faculty, staff, and graduate students conducted by the Affordability Task Force led to significant changes in faculty housing support, affordability enhancements for staff, and new funding guarantees for graduate students.
A major development in the use of assessment to improve teaching and learning at Stanford occurred when the university undertook an overhaul of course evaluations. A faculty committee leveraged quantitative analysis of existing course evaluation data and a structured focus group process to develop a new evaluation form centered on student learning. The revised forms, rolled out in Autumn of 2015-16, invite faculty to articulate course-specific learning outcomes. The goal of the revised forms are to provide better information to faculty, encourage students to reflect more thoughtfully about their educational experiences, and enhance the learning partnership.
Central university offices provide support and guidance for schools, departments, and administrative offices in their use of institutional data to drive improvement. For example, the university has established Minimum Privacy Standards and IR&DS has published a guide on Risks and Best Practices for Working with Student Data. The provost charged the Student Data Oversight Committee which reviews faculty and staff requests for use of student data in research. Additionally, the Vice Provost for Undergraduate Education has a policy requiring that studies conducted by VPUE personnel using student data limit data collection burden for students and follow the ethical conduct of research guidelines of the Conference on College Composition and Communication. As the university community enthusiastically embraces the collection and use of data for improvement, we are investing in training about privacy, appropriate use, and the proper stewardship of university data.
CFR 4.4
The institution, with significant faculty involvement, engages in ongoing inquiry into the processes of teaching and learning, and the conditions and practices that ensure that the standards of performance established by the institution are being achieved. The faculty and other educators take responsibility for evaluating the effectiveness of teaching and learning processes and uses the results for improvement of student learning and success. The findings from such inquiries are applied to the design and improvement of curricula, pedagogy, and assessment methodology.
Stanford faculty drive inquiry into the effectiveness of teaching and learning practices through multiple mechanisms. First, faculty lead committees and task forces charged with examining educational effectiveness and the success of our learning objectives. Examples include a study currently being conducted to examine the effectiveness of Stanford’s set of writing requirements and opportunities, both in general education and in the major. The committee designing the writing study has leveraged data on Stanford students’ writing preparation and backgrounds, has commissioned reviews of best practice and current research into writing pedagogy, and is determining what evidence would best address the questions guiding their inquiry.
The Breadth Governance Board (BGB) is charged with ensuring that students meet the learning objectives of the Ways of Thinking, Ways of Doing breadth requirements. In 2020, for example, the BGB undertook a recertification of the Exploring Difference and Power requirement, to ensure that approved courses had adequate coverage of structures of difference in the context of power relations (see discussion in CFR 2.4).
A key example of faculty-driven inquiry into the effectiveness of teaching and learning was the reform of end-of-term course evaluations, described above under CFR 4.3.
A second major mode of inquiry by Stanford faculty into effective teaching practices is discipline-based educational research (DBER). Stanford faculty in schools and departments across the university specialize in researching effective teaching practices. A few examples of this work include:
- Senior Lecturer Lisa Hwang, Professor Carl Wieman, and colleagues have published work on authentic problem-solving in chemical engineering education (here and here).
Faculty, instructors, and researchers in the department of Biology and the Graduate School of Education have published on inquiry-based teaching and scientific thinking in biology classes. The synergy between discipline-based educational research and faculty disciplinary research in this work was described in a Science article by Professor Tadashi Fukami.
CFR 4.5
Appropriate stakeholders, including alumni, employers, practitioners, students, and others designated by the institution, are regularly involved in the assessment and alignment of educational programs.
Stanford regularly engages stakeholders as part of assessment and feedback of educational programs. All bachelors, master’s, and PhD students are systematically surveyed at the time of graduation, providing rich feedback on degree programs. Undergraduate alumni from cohort blocks are surveyed every four years and are asked questions regarding how well Stanford prepared them for their postgraduate careers and further education. Additional forms of stakeholder engagement vary across schools and degree programs, reflecting the diverse missions of our educational programs. Below are examples from across Stanford’s schools of the ways that stakeholders are regularly involved in the assessment and alignment of educational programs:
- Interdisciplinary PhD programs in the School of Medicine comprehensively survey the current student and alumni populations of programs undergoing review. Questions include assessments of the program, curriculum, and career decision-making. Additionally, town halls are held with current students and affiliated program faculty, discussing key strengths, weaknesses, and areas for improvement. This input informs review findings.
- The Board of Visitors to the Stanford Law School regularly provides input to the dean on curricular reforms and the degree to which SLS graduates are adequately prepared for a variety of careers. Members of the Board of Visitors are employers and practitioners in legal, business, and public service sectors.
- The Stanford Teacher Education Program (STEP) in the Graduate School of Education engages with practitioner and employer partners through purposeful dialogue between STEP, schools, and school districts where teacher candidates are placed. Preparing teachers is the joint work of the university and these schools. There are regular opportunities for teachers and administrators at school sites to provide feedback to the STEP program. Additionally, Stanford has established close relationships with particular partner schools and the San Francisco Unified School District. Stakeholders from partner schools provide program feedback during school site meetings and frequent interactions during site visits.
In the School of Engineering, programs seeking ABET accreditation regularly survey students, alumni, graduate programs, and employers. These surveys include ratings of the relevance of learning outcomes, (indirect) assessment of the degree to which program graduates meet core learning outcomes, and gather general program feedback.
CFR 4.6
The institution periodically engages its multiple constituencies, including the governing board, faculty, staff, and others, in institutional reflection and planning processes that are based on the examination of data and evidence. These processes assess the institution’s strategic position, articulate priorities, examine the alignment of its purposes, core functions, and resources, and define the future direction of the institution.
Governance at Stanford is supported by a number of Faculty Senate and university committees at various levels of the institution. These committees, which include faculty, student, and staff representation, are regularly involved in shaping university priorities and planning. In addition to involvement in through university committees, student leadership regularly meet with the president, provost, and the vice provosts for student affairs, graduate and undergraduate education. Furthermore, many units across Stanford regularly engage advisory groups to ensure alignment of priorities. Advisory boards that support central offices include:
- VPUE
- Undergraduate Advisory Council (UGAC)
- Committee on Residential Learning (CoRL)
- Student Advisory Group (SAG)
- VPGE
- VPSA
In 2017, Stanford began a planning process to develop a shared vision for its future. This process culminated in the university's Long Range Vision. During the 2017-18 academic year, Stanford asked faculty, undergraduate students, graduate students, postdocs, staff, academic staff, and alumni for input. This generated more than 2,800 ideas, which were then distilled by committees into 37 white papers. Stanford’s Executive Cabinet formed cross-campus design teams that met throughout the 2018-19 academic year to develop recommendations for action which are now being refined and implemented.
Resource and capital planning are aligned with educational priorities as part of the annual budget process that is led by the provost (CFR 3.4).
CFR 4.7
Within the context of its mission and structural and financial realities, the institution considers changes that are currently taking place and are anticipated to take place within the institution and higher education environment as part of its planning, new program development, and resource allocation.
The planning process described in 4.6 above yielded several institutional priorities. New areas for scholarship were identified and provided with resources. These include research initiatives such as the Stanford Impact Labs, Human-centered Artificial Intelligence Institute, and a focus on climate and sustainability. The latter resulted in the launch of the Stanford Doerr School of Sustainability, which absorbed the Stanford School of Earth, Energy, and Environmental Science, and has embarked on new directions in research and education.
The same planning process launched several affordability initiatives for all members of the community. Recently announced enhancements include health insurance subsidy for graduate students, increased family grant program for graduate students and postdoctoral scholars, pilot programs for transitional housing for postdocs, one-time grants for early career faculty, and enhancements to faculty housing assistance.
Finally, new initiatives on student residential communities and a university town center also came out of the planning process (see CFR 2.13). Undergraduate Residential Education was reorganized and launched in fall 2021. The reimagined structure is around residential neighborhoods. Students live in one neighborhood throughout their four years at Stanford to create a continuous sense of community and foster a sense of belonging. Campus residential life for graduate students has also been transformed by the addition of on-campus housing for more than 2,400 graduate students at Escondido Village Graduate Residences.
In addition, annual planning processes such as the budget process and the capital planning process take into account priorities and needs for teaching and research (see CFR 3.4).