Continuous Improvement of Student Learning: Designing a Comprehensive Learner Logic Model
| Our Partner: University of Maryland, Baltimore County Location: Baltimore, MD The case study was written by Collin Sullivan, Jalisa Monroe of UMBC, and Suzanne Carbonaro of AEFIS. |
Introduction
COVID-19 exposed a number of instructional strategies and learning improvements necessary to support and measure student success. The trend is uniquely moving toward a buyers market for education—learners looking for an education that fits their circumstances at an affordable price. In the midst of this trend, the American Association of Colleges and Universities (AAC&U) 2021 workforce report highlights the importance of an alignment between institutional outcomes and skills to promote workforce preparedness, national competitiveness, and socioeconomic growth. Specifically, the report found “substantial support among employers for the outcomes and experiences of a liberal education and makes it clear that a liberally educated graduate is strongly positioned for success in the job market.”
With an opportunity to improve and innovate student learning outcomes simultaneously, Comprehensive Learner Record (CLR) is an interoperable achievement hub that reinforces that learning happens everywhere with verified evidence of student learning from course, co-curricular engagements, student employment, and even student self-identified experiences. CLR makes achievements explicit and meaningful for institutions, students, and employers. Stakeholders who use CLR as a vehicle for learning in real-time and an ecosystem of skills that employers value are providing opportunities to share knowledge, skills, and abilities with a greater audience.
What is a Comprehensive Learner Record?
According to Educause (2019), Comprehensive Learner Record “is a digital asset that helps students both better understand their learning and share a verifiable record of their knowledge and accomplishments. With a learner’s consent, the CLR gathers data about performance beyond just course grades, with an ultimate goal of capturing, recording, and communicating learning when and where it happens across a student’s higher education experience.”
IMS Global Learning Consortium has been working closely with its higher education, K-12 and educational technology partners over the last five years to define a standard for CLR that provides a “modern and web-friendly interoperable learner record structured for easy understanding yet flexible enough to support a wide range of use cases to meet the needs of learners and workers, registrars and employers.”
In early 2021, IMS Global released the first version of the CLR standard which has enabled institutions to choose ed tech providers who have attained certification and can deliver a fully functional CLR enabling portability of verifiable records for students to share digitally with employers.
This case study provides the context of the design, implementation and results of the University of Maryland, Baltimore County’s CLR infrastructure—their CLR logic model and how it is supporting curriculum revision, student learning, and preparing students for their future careers. A key component of this model is their partnership with the Greater Washington Partnership (GWP) a civic alliance of higher education and employers in the region from Baltimore, MD to Richmond, VA.
University of Maryland, Baltimore County (UMBC)
Founded in 1966, The University of Maryland, Baltimore County is one of twelve (12) member institutions of the University System of Maryland and is located in the suburban area surrounding Baltimore City. UMBC offers 48 undergraduate, 36 master, and 24 doctoral programs and serves approximately 14,000 students as of Fall 2021.
Skills-based learning at UMBC
As a leader in teaching and innovation by U.S. News & World Report, UMBC recognizes the importance of capturing knowledge, skills, and abilities students acquire in co-curricular and curricular endeavors that are not being captured in any identifiable or verifiable way (Braxton, Sullivan, Wyatt, & Monroe, 2022). Skills-based learning affords students at UMBC to share verifiable learning experiences to various channels including LinkedIn and other employer platforms and also allows them to articulate their expertise gained from experiential learning that is relevant for the workforce. Institutions thereby can determine the impact of extracurricular activities on graduation rates and post-graduation success. With these goals in mind, and building upon the foundation of Massive Open Online Courses (MOOCs) and Small Private Online Courses (SPOCs), UMBC is approaching learning modules in a more pragmatic and career-oriented model.
Badging Workflow
To support the design, development, and implementation of digital badges across the institution, the Instructional Technology team at UMBC developed a workflow for intake and requirements gathering; initial design meetings; badge design reviews; badge metadata, criteria, and assessment development; badge implementation within the badging platform, and badge awarding to earners (Braxton et. al, 2022). This workflow is presented in Figure 1.
To begin the workflow process, programs interested in creating a badge or system of badges must submit a help desk request through UMBC’s ticketing system. Once the request has been reviewed, an initial meeting is arranged with stakeholders to define the scope of the prospective badge and identify the badge team. The members of the badge team consist of Content, Technical, and Visual owners. After the badge team is assembled, metadata for the digital badge is collected by the content and technical owners. This includes information such as the badge name, description, criteria, evidence, the amount of time it will take to earn the badge and requirements.
The visual owner must also have an understanding of the badge purpose to best represent the badge in the design process. The visual owner should consider branding guidelines while crafting the badge design. Once the badge metadata and visual graphic are collected, the badge is built on an open badge platform. Prior to issuing the badge, the content and technical owners will draft a distribution process to identify how earners will be notified of the badge, when the badge will be issued, and where earners can locate the badge. A draft notification is also crafted to alert earners of their achievement, the day to expect the badge, and the company sending the badge. Upon confirmation of the date and time, the badge is issued to individuals verified as completing all badge requirements and demonstrating mastery of the related competency. Recipients receive an email with instructions on how to claim and share the badge on platforms such as LinkedIn, email signatures, and other websites using a shared link.
The CLR Logic Model
AEFIS enables higher ed leaders to create an end-to-end Comprehensive Learner Record as a result of their intentional programmatic curriculum mapping and outcomes alignment. The software gives program coordinators and CLR leaders like UMBC the space to create curricular outcomes and align these to institutional and program outcomes, strategic planning goals, skills frameworks, and accreditor standards.
Before embarking on the digitization of curriculum inside software, it is important to better ascertain your culture for CLR. Finley (2019) suggests using a logic model approach in conjunction with an assessment plan before embarking on high-impact practices to yield better results holistically. She writes,
“Finding the administrative resources and political capital to reach appreciable levels of scaled practices, along with sustainability of resources over time, are often the result of having sufficient evidence to make the case that such things are worth the time and investment.”
(Finley, 2019, p. 4)
The Comprehensive Learner Record Logic Model (Figure 2) takes into account these principles for a successful and sustainable undertaking of a CLR initiative that engages and impacts multiple stakeholders of an institution. In a recent roundtable presentation for IMS Global Learning Consortium (Abras, C., D’Aoust, R., Huckett, P., & Schreck, J., 2021), Janet Schreck, Associate Vice Provost for Education at Johns Hopkins University stated that
“CLR is conceptually dependent upon robust and authentic learning assessment. For that reason, CLR efforts can only succeed if faculty are involved as key stakeholders and partners in the vision.”
Janet Schreck, Associate Vice Provost for Education at Johns Hopkins University
Beginning with a discussion of the impact of a CLR initiative with key stakeholders provides an opportunity to align an institution’s mission and vision with both the process and outcomes of this undertaking (Braxton, S. N., Carbonaro, S., & Jankowski, N. (2022).
Next, stakeholders need to unpack the outcomes and expectations of CLR. Some key essential questions include:
- What will success look like in the short, intermediate and long-term?
- What data will be collected to review and revise to determine success? This discussion will likely lead to identifying activities to include in the CLR.
- Where do these activities live?
- What data are available on the activities?
- What outcomes will be linked to these activities and at what level of learning?
- Who are the stakeholders of CLR? How will they contribute to this initiative?
- What evidence will be captured in a CLR and who will verify it?
Finally, institutions need to assess their resources which includes budget, human capital, software, and time. Identifying the right stakeholders in this process early and providing insight into what their workload will be with regards to learning the software, tasks for completion, and time frames are key to the success of a CLR initiative. Table 1 represents how UMBC structured their needs and human resources for CLR.
| Needs | Stakeholders |
| Institutional Mission/Vision Alignment | Academic Affairs (Provost, Institutional Effectiveness) |
| Outcomes and Expectations of CLR | Program Coordinators, Faculty, Student Affairs, Experiential Learning, Employers |
| Learning Activities and Experiences to include in CLR | Assessment leaders, instructional technology, registrar and departmental coordinators of assessment and activities to liaise with faculty and staff, including deans and department chairs. |
| Clear CLR Communication Plan for faculty, staff and students | Student leaders, department chairs and deans, assessment leaders |
ROI for Learners
The skills-based learning trend is uniquely moving toward a buyers market for education—learners looking for an education that fits their circumstances at an affordable price. Learners are able to identify competencies or skills they need for their careers or other pursuits, earn verifiable credentials to demonstrate mastery, and finally display it through a Comprehensive Learner Record.
The Greater Washington Partnership (GWP) is a civic alliance of the region’s leading employers who are committed to fostering unity and making the Capital Region—from Baltimore to Richmond—the most inclusive economy in the nation. GWP offers Digital Tech Credentials which allows the region’s leading businesses and academic leaders to collaborate in scaling a coordinated response to pressing workforce needs in the areas of data and analytics, probability and statistics, data manipulation, data visualization, data ethics, and data security.
UMBC is a leader in the Greater Washington Partnership aligning a number of modules and courses to the GWP framework for the Digital Generalist and Specialist credentials. Early findings reveal that learners, students from both STEM and non-STEM backgrounds, are taking advantage of the coursework and modules being offered to earn the micro-credential. Currently, badges are being issued to the learners externally but as a partner of AEFIS, UMBC will begin to enable the badge and its metadata to flow to a student’s CLR. The CLR infrastructure for this goal is underway with learners beginning to see their CLR in early spring 2022. AEFIS is the first ed tech company to attain IMS CLR certification and UMBC will be one of the first institutions to use the new CLR platform. In addition, AEFIS holds IMS Global Open Badges 2.0 certification which enables institutions to issue badges through AEFIS or ingest them from other third party certified vendors into AEFIS.
As UMBC looks ahead to the future, a number of additional certificate programs, undergraduate and graduate programs, co-curricular, and extra-curricular activities are underway to be included into the CLR logic model.
References
Abras, C., D’Aoust, R., Huckett, P., & Schreck, J. (2021, April 19). Tips for collaborating across your institution to achieve success with CLR [Webinar]. IMS Global Learning Consortium Roundtables. https://www.youtube.com/watch?v=F5omEeisyNo
Braxton, S. N., Carbonaro, S., & Jankowski, N. (2022). Comprehensive Learner Record as a Vehicle for Assessment and Learning Transparency in a Skills Economy. In Y. Huang (Ed.), Handbook of Research on Credential Innovations for Inclusive Pathways to Professions (pp. 214-233). IGI Global. http://doi:10.4018/978-1-7998-3820-3.ch011
Braxton, S. N., Sullivan, C., Wyatt, L. A., & Monroe, J. (2022). Capturing Student Achievement and Learning Pathways at the University of Maryland, Baltimore County: Digital Badging and the Comprehensive Learner Record. In Y. Huang (Ed.), Handbook of Research on Credential Innovations for Inclusive Pathways to Professions (pp. 396-417). IGI Global. http://doi:10.4018/978-1-7998-3820-3.ch020
Carbonaro, S. (2020). Comprehensive learner record: Exploring a new transcript for lifelong learning. IMS Global Learning Consortium edTech rEvolutions leaders Retrieved from: https://www.imsglobal.org/article/edtech-leaders/clr
Comprehensive Learner Record. (n.d.). IMS Global Learning Consortium.http://www.imsglobal.org/activity/comprehensive-learner-record
Dennen, V. P. & Burner, K. J. (2007). Instructional strategies and models associated with cognitive apprenticeship. Handbook of Research on Educational Communications and Technology, (pp. 425-439). Retrieved from: https://pdfs.semanticscholar.org/ecc1/b2df2d37f995739986739397829f7e7ff4d9.pdf
EDUCAUSE Learning Initiative (2019). Seven things you should know about: The comprehensive learner record. Retrieved from: https://library.educause.edu/-/media/files/library/2019/1/eli7164.pdf
Finley, A. (2021). How college contributes to workforce success: Employer views on what matters most. Washington, DC: Association of American Colleges and Universities.
Finley, A. (2019, November). A comprehensive approach to assessment of high-impact practices (Occasional Paper No. 41). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment.
University of Maryland, Baltimore County. (n.d.). https://www.umbc.edu/
University of Maryland, Baltimore County – Greater Washington Partnership / Capital CoLAB. (n.d.). https://gwp.umbc.edu/
