Blended Teaching Readiness: Phase 2 – Instrument Development

Written By:
Charles R. Graham, Brigham Young University
Jered Borup, George Mason University
Emily Pulham, Brigham Young University
Ross Larsen, Brigham Young University

Suggested Citation

Graham, C. R., Borup, J., Pulham, E., & Larsen, R. (2018). K-12 blended teaching readiness: Phase 2 – instrument development. Lansing, MI: Michigan Virtual University. Retrieved from https://mvlri.org/research/publications/blended-teaching-readiness-phase-2-instrument-development/

This research completes a two-year research process to create and empirically validate an instrument to measure K-12 Blended Teaching Readiness. This report details the process as well as the successful efforts to validate and make available an instrument for use by individuals, schools, districts, and universities. Additionally, the report documents the process of creating a second, shorter instrument focusing on four of the most essential pedagogical competencies for blended teaching. In addition to the development of readiness instruments, we created a guidebook for open and free use titled, K-12 Blended Teaching: A Practical Guide for Teachers

Executive Summary

Blended teaching involves combining online and in-person instruction.  There has been significant growth in the demand for blended learning in schools which has increased the need for teachers who know how to teach in a blended learning environment.

In Phase 1 of the study, a blended teaching readiness instrument containing four dimensions was tested and statistically validated (see Graham, Borup, Pulham, & Larsen, 2017).  We determined that the instrument was missing an important dimension related to management of the space and routines within the blended environment.  As a result, in Phase 2, the subject of this report, we added a fifth dimension of Management to the instrument for a total of 65 items.  We report here on the completed instrument and the work to have experts in the field, administrators, and teachers review the new items prior to testing it statistically with over 250 teachers.  The domains include the following five top-level constructs, each containing 2-4 sub-constructs:

  1. Foundations
    1. Technological Literacy
    2. Digital Citizenship
    3. Dispositions
  2. Planning
    1. Planning Blended Activities
    2. Planning Blended Assessments
  3. Instructional Methods & Strategies
    1. Personalized Instruction
    2. Facilitating Student-Student Interaction
    3. Facilitating Student-Instructor Interaction
    4. Facilitating Student-Content Interaction
  4. Evaluation & Assessment
    1. Implementing Blended Assessments
    2. Evaluating and Reflecting
  5. Management
    1. Managing Blended Environments
    2. Managing Blended Routines

Additionally, we report on the early development of a shorter instrument focused specifically on teacher dispositions and the following five key pedagogical competencies:

  1. Dispositions – beliefs and attitudes related to blended teaching and learning.
  2. Online Integration – ability to effectively combine online instruction with in-person instruction.
  3. Real-time Data Practices – ability to use digital tools to monitor student activity and performance in order to guide student growth.
  4. Personalizing Instruction – ability to implement a learning environment that allows for student customization of goals, pace, and/or learning path.
  5. Online Interaction – ability to facilitate online interactions with and between students.

While there is overlap between the two instruments, the shorter instrument organizes the blended teaching constructs in a way that we believe will be simpler for administrators and teachers to build into focused professional development.  We created the readiness instruments and a professional development guidebook on blended teaching competencies for open and free access by teachers, districts, and university educator preparation programs (See K-12 Blended Teaching: A Practical Guide for Teachers ).

Introduction and Background

As the demand for quality, flexible K-12 educational options increases, there is a growing need for teachers who are prepared to teach effectively in blended learning classrooms that combine elements of in-person and online instruction (Pulham & Graham, 2018; Pulham, Graham, & Short, 2018).  At the same time, the demands on teacher time and the needs for professional development are as high as they have ever been (Pulham & Graham, 2018; Pulham, Graham, & Short, 2018).  The purpose of both Phase 1 and Phase 2 of this project has been to help facilitate and evaluate this process by developing a blended teaching readiness instrument that can,

  • help educators know what specific skills are necessary for success in a blended environment,
  • allow educators to evaluate their readiness for teaching in a blended environment,
  • provide information that would allow for a personalized approach to teacher professional development for acquiring blended teaching skills, and
  • measure teachers’ change in readiness following targeted professional development.

This research and development has been conducted over a two-year period.  The 2017 MVLRI Phase 1 Report contains an overview and study details from Phase 1 of the project (Graham, Borup, Pulham, & Larsen, 2017).  Additionally, since that time, we have published two papers that do a systematic synthesis of the existing literature related to K-12 blended teaching competencies.

  • Comparing K-12 online and blended teaching competencies: A literature review (Pulham & Graham, 2018)
  • Generic vs. Modality-Specific Competencies for K-12 Online and Blended Teaching (Pulham, Graham, & Short, 2018)

The review of existing literature provided a foundation for the development of the blended teaching competency structures that are described in this report.  Readers are invited to review the resources linked above for a comprehensive overview of the project.

During Phase 2 of the project, we did the following:

  • Added a Management construct to the Blended Teaching Readiness instrument with sub-constructs of (a) managing the blended learning environment and (b) managing blended learning routines for a total of 65 items.
  • Solicited feedback on the new items from experts, district administrators, and teachers.
  • Administered the survey to 258 current teachers and performed statistical tests to evaluate the fit of the competency model used.
  • Developed a shorter instrument with 47 items focusing on four core pedagogical competencies needed for blended teaching: (1) online integration, (2) real-time data practices, (3) personalization, and (4) online interaction.
  • Solicited feedback from experts and teachers on the items.

Instrument Development

Blended Teaching Competency Models

The blended teaching competency model that was initially developed during Phase 1 of the research had two levels as shown in Figure 1.  This model is what we have begun to refer to as the process model because its top-level structure is process-oriented (e.g., planning, methods, assessment/evaluation, management).  At the second-level, you begin to see constructs specific to blended teaching.  The process model was quite comprehensive, but also made it hard to emphasize some important pedagogical concepts to blended teaching, like real-time data practices or personalization, that had elements distributed across multiple constructs.  Additionally, the complexity inherent in a model with five top-level constructs, 13 sub-constructs, and 65 items, introduced measurement challenges as well as challenges to designing professional development aligned with each of the constructs.

Figure 1 details the process model for blended teaching competencies. The model is a three-leveled hierarchical map with the first level on the left titled Blended Teaching (Process Model). Five second-level constructs are to the right of the first level and titled from top to bottom; they are, 1. Foundations, 2. Planning, 3. Instructional Methods & Strategies, 4. Assessment & Evaluation, and 5. Management. The third level in the map is to the right of second level and from top to bottom are, 1.1 Technological Literacy, 1.2 Digital Citizenship, 1.3 Dispositions, 2.1 Planning Blended Activities, 2.2 Planning Blended Assessments, 3.1 Personalized Instruction, 3.2 Facilitating Student-Student Interaction, 3.3 Facilitating Student-Instructor Interaction, 3.4 Facilitating Student-Content Interaction, 4.1 Implementing Blended Assessments, 4.2 Evaluating and Reflecting, 5.1 Managing Blended Environments, and 5.2 Managing Blended Routines.
Figure 1. Process Model for Blended Teaching Competencies.

In Phase 2 of the research, we continued to update and do the validation research for the process model, and we also conceptualized and began the development work for a pedagogical model that would focus on a small number of core blended teaching pedagogical skills.  While the pedagogical model would be less comprehensive, it would also be more focused and therefore more useful in a professional development context with limited time and resources.  Figure 2 shows the core constructs of the pedagogical model.

  • Dispositions – beliefs and attitudes related to blended teaching and learning.
  • Online Integration – ability to effectively combine online instruction with in-person instruction.
  • Real-time Data Practices – ability to use digital tools to monitor student activity and performance in order to guide student growth.
  • Personalizing Instruction – ability to implement a learning environment that allows for student customization of goals, pace, and/or learning path.
  • Online Interaction – ability to facilitate online interactions with and between students.
Figure 2 details the pedagogical model for blended teaching competencies. The model is a two-leveled hierarchical map with the first level titled Blended Teaching (Pedagogical Model). Five second-level constructs are titled from top to bottom, 0. Dispositions, 1. Online Integration, 2. Real-time Data Practices, 3. Personalizing Instruction, and 4. Online Interaction.
Figure 2. Pedagogical Model for Blended Teaching Competencies.

Instrument Development Procedures

The steps below were used in the development of both process and pedagogical instruments.  The steps were also used for the blended teaching process model in Phase 1 and then with the Management items again in Phase 2.  During Phase 1 we determined that the instrument was missing an important aspect of implementation which deals with classroom management of the learning environment as well as routines that are often disrupted when moving to a blended approach.  Because of timing issues with our district partners, we were not able to delay administration of the Phase 1 instrument for the amount of time that was required for proper vetting of new items.  Therefore, the Management section was developed using the same process outlined below added to the instrument during Phase 2 of the research. For development of the pedagogical model we did not get feedback from district administrators (step 5) due to logistical and time constraints.

  1. Reviewed existing frameworks – This consisted of the extensive analysis outlined previously along with familiarizing ourselves with the existing standards and frameworks related to blended learning and reviewing the competency items (Pulham & Graham, 2018; Pulham, Graham, & Short, 2018).
  2. Developed a competency structure – In this step, we sought a balance between comprehensiveness and parsimony of the model (Whetten, 1989). Ultimately the process model leaned towards comprehensiveness while the pedagogical model leaned towards parsimony.
  3. Developed items for constructs – In this step, we developed items for each of the constructs in the two models. As would be expected, there is some overlap between items in the two models.  We ended up with five items for each of the 13 constructs in the process model (65 total) and 8-11 items for each of the five constructs in the pedagogical model (47 items with the expectation of pruning the items even more after statistical tests were run).
  4. Solicited external expert feedback – In this step we sought feedback from four to six expert reviewers who are top scholars and leaders in the area of K-12 online and blended learning. We made adjustments to items based on their feedback.
  5. Solicited district administrator feedback – In this step, we received several rounds of feedback in both Phases 1 and 2 from a large school district’s blended and personalized learning instructional service team. We made adjustments to items based on their feedback.
  6. Solicited practicing teacher feedback – Finally, we held interviews with a handful of current teachers at elementary, middle, and high school levels who followed a think aloud protocol to react to each item in the instrument in order to identify confusing language or jargon. Several minor changes were made to make items easier to understand.

The result of this process can be found in the latest versions of the two blended teaching readiness instruments found in Appendix A.  In Phase 2, the process instrument went through another round of statistical analysis with the new Management constructs and items that will be described below.  While the shorter pedagogical model instrument has been developed, it has yet to be administered to teachers.  As a result, the following sections will focus on the methods and findings regarding the process model.

Methods

In Phase 2 of this research, the Management construct and two sub-constructs were added to the process model that was successfully tested for fit in Phase 1 (See Figure 3). This section describes the data collection and analysis procedures that were conducted to determine if the management item works with the model.

Figure 3 details the structural model for testing of the process model. The model is a three-leveled hierarchical map with five constructs in ovals across the top from left to right: 1. Foundations, 2. Planning, 3. Instructional Methods & Strategies, 4. Assessment & Evaluation, and 5. Management. Connected to each of the top-level ovals is a second level with numbers in ovals representing the second-level constructs and connected to those arrows to small squares, representing the items for each construct. Connected below 1. Foundations is 1.1, 1.2, and 1.3 each with five squares representing the number of items in the instrument for each construct. Connected below 2. Planning is 2.1, 2.2, each with five squares representing the number of items in the instrument for each construct. Connected below 3. Instructional Methods & Strategies is 3.1, 3.2, 3.3, and 3.4 each with five squares representing the number of items in the instrument for each construct. Connected below 4. Assessment & Evaluation is 4.1, 4.2 each with five squares representing the number of items in the instrument for each construct. Connected below 5. Management is 5.1, 5.2 each with five squares representing the number of items in the instrument for each construct.
Figure 3. Structural Model for Phase 2 Testing of the Process Model (see items in Table A1).

The participants for Phase 2 of the process model developed in this research were 258 in-service teachers in a district in the eastern United States.  We had broad representation across teaching levels and subject areas.  The majority of participants had little to no blended teaching experience.  Table 1 summarizes the demographic profile of the Phase 1 and Phase 2 samples.

Data Collection

Table 1. Demographic data for participants who completed the process model instrument
 

Demographic Variable

Number of Teachers
Phase 1
Number of Teachers
Phase 2
Total number of participants 218 258
Level taught
Grades PK-6 149 68
Grades 7-12 69 190
Secondary subjects taught
Science 21 35
Language arts 21 36
Special education 11 11
Social studies 8 24
Math 7 27
World languages 5 10
Career and technical education 4 9
Health and physical education 2 13
Performing and fine arts 2 5
Other 7 17
Years of PK-12 teaching experience
1-5 61 70
6-10 52 50
11-15 42 49
16-20 29 41
21+ 34 48
Years of blended teaching experience
0 108 92
1-5 91 127
6-10 13 24
11-15 5 10
16-20 1 5
21+ 0 0
Years of online teaching experience
0 213 254
1-5 4 4
6-10 1 0
11+ 0 0

Data Analysis

We ran a confirmatory factor analysis (CFA) on theoretical structure as represented in Figure 3.  The models were tested in Mplus using established cutoffs for four different fit statistics (RMSEA < .08, CFI > .9, TLI > .9, SRMR < .08; Wang & Wang, 2012).  Each fit statistic makes slightly different assumptions about the structural models; achieving the cutoff levels for more fit statistics indicates a stronger model.  In the case that the model did not fit one or more of the indices, we planned to test each of the top-level factors individually for structural fit using the same CFA process.

Results

Process Model Fit

The CFA for the full theoretical process model (Figure 3) met two of the four fit statistics thresholds set forth in the literature (see Table 2).  These results were not as good as the Phase 1 model which met all four fit statistics thresholds.  However, they are acceptable considering the additional complexity of the model and large number of parameters being estimated in the model.  Factor loadings and communalities were examined (see Table A1 in Appendix A), and all factor loadings were statistically significant at the 0.01 level.  Additionally, only three of 65 items had communalities that were below 0.5, and those three items were in the previously validated portion of the Phase 1 instrument and are theoretically important so we chose to keep them in the model. (Worthington & Whittaker, 2006).

Each superfactor CFA (e.g., foundations, planning, methods, assessment/evaluation, management) was run separately and the fit statistics calculated (Table 3) to isolate any local misfit in each superfactor.  The fit statistics for all the superfactors were within acceptable ranges (with exceptions of RMSEA for the management and assessment factors).  This result gives further evidence that the lack of fit for the theoretical model has to do with model complexity rather than any model misspecification. (Wang & Wang, 2012). Thus, we determined the model to be defensible.  This also lends weight to our decision to begin development of a model that is less complex with fewer constructs and no second-order factors.

Table 2. Fit statistics for the Phase 2 blended teaching readiness “process” model (𝝌2= 3746.282, df= 1992).
Variable CFI

> 0.9

TLI

> 0.9

RMSEA

< 0.08

SRMR

< 0.08

Value 0.875 0.875 0.058 0.056
Fit Poor Poor Good Good
Table 3. Fit statistics for each high-level factor in the confirmatory factor analysis (CFA) (n=258).
Superfactor CFA CFI
> 0.9
TLI
> 0.9
RMSEA
< 0.08
SRMR
< 0.08
1. Foundations 0.941 0.929 0.066 0.058
2. Planning 0.972 0.963 0.063 0.030
3. Pedagogy 0.922 0.911 0.078 0.056
4. Assessment 0.956 0.942 0.081 0.031
5. Management 0.956 0.940 0.092 0.027 

Discussion & Conclusion

This research will complete the validation work for a process model of the blended teaching readiness instrument.  We present data based on the first 258 respondents, which is a small N for the large number of items and constructs included in the model.  Nevertheless, we determined that the instrument was suitable for K-12 inservice teachers.  We plan to add an addendum to this report when additional data is received and expect that the additional data will add to the confidence we have in the validity of the instrument.  Validation work involved input from experts, district administrators, and practicing teachers, as well as statistical tests to determine the fit of the theoretical model.  Phase 2 of the research added 10 items related to managing a blended environment and blended routines.  These were deemed important additions recommended in Phase 1.  However, we were a little disappointed that while deemed adequate, the statistical fit of the new model was not as strong as the Phase 1 model.  There are many potential reasons for this fact.  The most likely reason is that the model was quite complex with five high-level constructs and 13 sub-constructs, and, at the time of this report, we only had 258 respondents.  A model of this complexity is quite challenging to test and typically requires a larger number of participants than we had access to.

In our initial work we opted for creating a model that tried to be fairly comprehensive from planning to evaluation.  However, we also recognized that the length, structure, and complexity of the process model instrument could limit its utility in some contexts.  We felt confident that we could capture the most essential skills in a simpler pedagogical model that focused on a smaller and highly relevant set of competencies.  We have presented these in this report as the pedagogical model.  We have used experts and teachers to help us develop items for a simplified blended teaching readiness instrument.  Future research will take the pedagogical model instrument through the statistical tests to further establish its validity for use with inservice as well as preservice teachers.

An important goal of this project has been to create tools and resources for measuring K-12 blended teaching readiness that are openly and freely available to individuals, school, districts, and universities who are trying to improve the blended teaching skills of K-12 teachers.  We feel that we have accomplished this important goal and we continue to do scholarly work that will advance this foundational research.

Below are the current resources that are available to you for measuring and developing K-12 blended teaching competencies.  Some possibilities for use of these resources include:

  • A school district could administer the blended teaching readiness survey to teachers across the district to determine schools that are most prepared to begin a blended learning initiative or to identify teachers in each school who can be recruited to be prepared as blended teaching coaches for their schools.
  • Teachers in a school take the blended teaching readiness survey as a pre-assessment, and then professional development activities using the guidebook are focused on greatest areas of need. Teachers could also be surveyed following the professional development to measure growth.
  • Individual teachers wanting to learn more about blended teaching and upgrade their skills in this area could use the blended teaching readiness survey as a self-assessment and then use the guidebook as a source of classroom examples and implementation ideas in each blended teaching competency area.
Resource Description
Blended Teaching Readiness Instrument (Process Model – 65 Items)

http://bit.ly/BTRSurvey1

Click on the survey link to view the survey.  At the end of the survey you will receive (1) a score in each of the competency areas and (2) a set of links to existing training resources related to the competencies. You may also use this instrument freely, in your own way, under a CC-BY license.
Blended Teaching Readiness Instrument (Pedagogical Model)

http://bit.ly/BTRSurvey2

This is the link to the shorter, more focused instrument. (Validation work is ongoing.)  At the end of this instrument, you will also have the option of having scores and training resources emailed to you.  Additionally, the core competencies outlined in this instrument are directly connected to the free OER professional development resource, K-12 Blended Teaching: A Practical Guide for Teachers.
K-12 Blended Teaching: A Practical Guide for Teachers

http://bit.ly/K12-Blended-Teaching-Book

This free OER guidebook was developed to help teachers develop the core skills needed to teach in environments that blend online and in-person learning.  It is expected that the guidebook will be publicly available in January 2019 (contact author Charles Graham if you would like to receive and review an early copy).

References

Graham, C. R., Borup, J., Pulham, E., & Larsen, R. (2017). K-12 blended teaching readiness: Phase 1 – instrument development. Lansing, MI. Retrieved from https://mvlri.org/wp-content/uploads/2017/11/k12-blended-teaching-readiness-phase-1-instrument-development.pdf

Pulham, E. B., & Graham, C. R. (2018). Comparing K-12 online and blended teaching competencies: A literature review. Distance Education, 39(3), 411–432. http://doi.org/10.1080/01587919.2018.1476840

Pulham, E., Graham, C. R., & Short, C. R. (2018). Generic vs. Modality-Specific Competencies for K-12 Online and Blended Teaching. Journal of Online Learning Research, 4(1), 33–52. Retrieved from https://www.learntechlib.org/j/JOLR/v/4/n/1/

Wang, J., & Wang, X. (2012). Structural equation modeling: Applications using Mplus. West Sussex, UK: John Wiley & Sons.

Whetten, D. A. (1989). What constitutes a theoretical contribution? The Academy of Management Review, 14(4), 490–495. Retrieved from https://www.jstor.org/stable/258554

Worthington, R. L., & Whittaker, T.A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806-838.

Appendix – Constructs and Items for Instrument

Table A1. Confirmatory Factor Analysis results for blended teaching readiness model (n =258 teachers).
Item Λ S.E. Std. λa Comm.b
1.1 Technical Literacy        
1.1.1: Master new online technologies on your own. 1.000 NA 0.836 0.698
1.1.2: Successfully troubleshoot unfamiliar technological issues that you and students encounter. 0.982** 0.054 0.802 0.643
1.1.3: Use the tools commonly found in a learning management system (e.g., gradebook, announcements, content pages, quizzes, discussion boards). 0.841** 0.073 0.762 0.581
1.1.4: Use content-specific educational software outside of the learning management system (e.g., math/literacy/science educational software, educational games). 0.874** 0.083 0.756 0.572
1.1.5: Find quality online content resources relevant to student learning needs (e.g., media resources, lesson plans, etc.). 0.627** 0.076 0.602 0.362
1.2 Digital Citizenship        
1.2.1: Model the legal use of instructional materials (e.g. copyright, fair use, creative commons). 1.000 NA 0.713 0.508
1.2.2: Ensure student online privacy (e.g., technology use agreements for sharing student data, protection of online data and identities). 1.31** 0.103 0.859 0.737
1.2.3: Model online safety for students (e.g., ensure password protection, protect against cyberbullying, detect scams, use content filters and virus software, etc.). 1.298** 0.104 0.848 0.720
1.2.4: Ensure academic honesty in an online learning environment (e.g., prevent cheating, check for plagiarism, etc.). 1.052** 0.127 0.731 0.535
1.2.5: Ensure access to online learning activities for all students (e.g., low socioeconomic status, English language learners, special education, gifted, etc.). 0.935** 0.125 0.664 0.440
1.3 Dispositions        
1.3.1: I believe students perform better when they have some control over the pace of their learning. 1.000 NA 0.643 0.413
1.3.2: I believe individual student access to devices in the classroom should enable students to take greater ownership of their learning. 1.432** 0.177 0.843 0.711
1.3.3: I believe online technologies allow students and teachers to do things that would be difficult or impossible in classrooms without online technologies. 1.333** 0.182 0.832 0.693
1.3.4: I believe it is important for teachers to explore new teaching strategies that blend in-person and online learning. 1.252** 0.180 0.811 0.658
1.3.5: I believe individual student access to online devices in classrooms enables development of important life skills (e.g., creativity, collaboration, critical thinking, communication). 1.553** 0.204 0.866 0.750
2.1 Planning Blended Activities        
2.1.1: Create activities that combine online and in-person components to help students develop important life skills (e.g., creativity, critical thinking, communication, and collaboration). 1.000 NA 0.865 0.749
2.1.2: Sequence activities in the learning management system in an easy-to-follow format. 0.927** 0.056 0.807 0.651
2.1.3: Strategically combine online and in-person activities that enable student ownership of their learning (e.g., flexibility in when, where, and how they learn). 1.082** 0.044 0.894 0.800
2.1.4: Incorporate existing online and offline educational materials into learning activities. 0.877** 0.061 0.811 0.657
2.1.5: Create new online learning materials when relevant content is not available. 1.026** 0.049 0.818 0.670
2.2 Planning Blended Assessments        
2.2.1: Create performance-based assessments that require students to use technology in ways that demonstrate important life skills (creativity, critical thinking, communication, collaboration). 1.000 NA 0.839 0.704
2.2.2: Create formative assessments to measure students’ learning progress (e.g., polls, online surveys). 1.018** 0.066 0.797 0.636
2.2.3: Incorporate appropriate media into assessments (e.g., video, audio, images). 0.947** 0.059 0.768 0.589
2.2.4: Determine when to use computer-administered vs paper-based assessments. 0.885** 0.056 0.777 0.604
2.2.5: Create an approach to assessment that allows for student choice in how they demonstrate mastery of learning objectives. 0.929** 0.053 0.777 0.604
3.1 Personalizing Instruction        
3.1.1: Use data collected online to customize students’ learning experience. 1.000 NA 0.737 0.543
3.1.2: Use data collected online to determine which groups or individual students need additional instructional support. 1.086** 0.052 0.797 0.635
3.1.3: Answer students’ course related questions online (in addition to in-person). 1.128** 0.102 0.788 0.620
3.1.4: Use student performance data to provide timely help with misconceptions. 1.075** 0.078 0.868 0.753
3.1.5: Address any limitations of educational software through individual or small group instruction. 1.106** 0.102 0.785 0.616
3.2 Facilitating Student-Student Interaction        
3.2.1: Facilitate students’ small group discussions online (in addition to in-person). 1.000 NA 0.887 0.787
3.2.2: Facilitate students’ small group collaboration on projects online (in addition to in-person). 1.031** 0.033 0.905 0.819
3.2.3: Strengthen students’ sense of belonging to the classroom community using online communication. 0.970** 0.037 0.880 0.775
3.2.4: Monitor students’ online interactions with each other to ensure quality participation. 1.052** 0.036 0.895 0.801
3.2.5: Create opportunities for students to teach each other inside and outside of class using online technology. 1.014** 0.047 0.858 0.736
3.3  Facilitating Teacher-Student Interaction
3.3.1: Determine when it is most effective to interact with students online versus in-person. 1.000 NA 0.773 0.598
3.3.2: Strengthen caring relationships with students via online communication. 1.232** 0.071 0.894 0.800
3.3.3: Convey your personality in online text-based communication with students. 1.239** 0.082 0.879 0.772
3.3.4: Ensure students are comfortable communicating with you online. 1.232** 0.079 0.909 0.826
3.3.5: Promptly respond to student inquiries online (in addition to in-person). 1.066** 0.086 0.798 0.636
3.4 Facilitating Student-Content Interactions        
3.4.1: Ensure that students can navigate online educational materials. 1.000 NA 0.844 0.713
3.4.2: Use the learning management system to monitor student activity with online educational materials to determine if they are on-task. 1.117** 0.079 0.772 0.596
3.4.3: Use data to monitor student progress in subject-specific software programs. 1.018** 0.078 0.770 0.593
3.4.4: Help students to select online and offline materials that are relevant to them. 1.071** 0.066 0.855 0.731
3.4.5: Encourage student persistence with independent online learning activities (in addition to in-person activities). 1.134** 0.067 0.851 0.725
4.1 Implementing Blended Assessments        
4.1.1: Administer performance-based assessments online (in addition to offline assessments). 1.000 NA 0.790 0.623
4.1.2: Use online tools to provide students with opportunities for reflective self-assessment. 1.071** 0.053 0.83 0.689
4.1.3: Use online and traditional grading rubrics to clearly identify individual student performance gaps. 1.039** 0.063 0.821 0.674
4.1.4: Use data from online and offline assessments to identify patterns in group and whole class learning gaps. 0.952** 0.066 0.767 0.588
4.1.5: Help students use online and offline assessment data to guide their own learning progress. 1.057** 0.064 0.855 0.731
4.2 Evaluating and Reflecting        
4.2.1: Use student performance data to evaluate the effectiveness of teachers’ online instruction. 1.000 NA 0.872 0.760
4.2.2: Use student performance data to evaluate the effectiveness of online educational materials and assessments. 1.072** 0.037 0.924 0.854
4.2.3: Use student performance data to evaluate the effectiveness of how online and in-person activities and assessments were blended together. 1.066** 0.047 0.898 0.806
4.2.4: Provide students with multiple opportunities to provide input about the effectiveness of the online and in-person teaching strategies. 1.036** 0.050 0.876 0.767
4.2.5: Collaborate with other teachers to evaluate the effectiveness of units that blend online and in-person instruction. 0.994** 0.063 0.823 0.677
5.1 Managing the Blended Learning Environment
5.1.1: Configure the classroom space as needed to support the planned in-person and online classroom-based activities. 1.000 NA 0.833 0.694
5.1.2: Develop strategies for organizing and maintaining online learning materials (e.g., online documents, hyperlinks, resources, etc.). 1.073** 0.050 0.878 0.771
5.1.3: Develop procedures for the online submission and management of student-created products (e.g., projects, reports, assignments, etc.). 1.031** 0.051 0.885 0.783
5.1.4: Establish clear procedures to help students manage the use of individual and/or classroom devices (laptops, tablets, headphones, etc.). 0.963** 0.049 0.850 0.723
5.1.5: Help students to manage their class-related online accounts and passwords. 0.986** 0.049 0.852 0.726
5.2 Managing Blended Learning Routines
5.2.1: Provide clear classroom procedures for transitioning between online and in-person learning activities. 1.000 NA 0.907 0.823
5.2.2: Establish procedures for how students should seek help when learning with online technology. 0.988** 0.034 0.914 0.836
5.2.3: Manage a classroom where students pursue mastery at their own pace. 0.903** 0.046 0.822 0.675
5.2.4: Establish guidelines that help students use online time wisely. 0.978** 0.039 0.905 0.819
5.2.5: Constructively intervene to address student use of technology that interferes with learning. 0.961** 0.044 0.870 0.757
Superfactor 1. Foundations
Technical Literacy 1.000 NA 0.841 0.708
Digital Citizenship 0.699** 0.091 0.673 0.453
Dispositions 0.491** 0.089 0.605 0.366
Superfactor 2. Planning        
Planning Blended Activities 1.000 NA 0.929 0.863
Planning Blended Assessments 1.039** 0.058 0.959 0.921
Superfactor 3. Instructional Methods & Strategies        
Personalizing Instruction 1.000 NA 0.890 0.792
Facilitating Student-Student Interaction 1.272** 0.106 0.871 0.759
Facilitating Teacher-Student Interaction 1.061** 0.105 0.858 0.737
Facilitating Student-Content Interaction 1.179** 0.101 0.966 0.933
Superfactor 4. Assessment & Evaluation        
Implementing Blended Assessment 1.000 NA 0.984 0.968
Evaluating and Reflecting 1.038** 0.074 0.920 0.846
Superfactor 5. Management
Blended Learning Environment 1.000 NA 0.967 0.935
Blended Learning Routines 1.018** 0.047 0.946 0.896
**  p < .01. aStandardized factor loadings based on Mplus 8.1 output. b
Table A2. Items for the pedagogical model of the blended teaching readiness instrument.
Constructs and Items
0. Dispositions (Rate your agreement with the following beliefs . . . )

0.1 I believe most students have better learning outcomes when technology enables them to adjust the pace of their own learning.

0.2 I believe that online technology is critical to implementing mastery-based learning in the classroom.

0.3 I believe online technologies enable valuable activities that would be difficult or impossible to do otherwise.

0.4 I believe it is important for teachers to explore new teaching strategies that blend in-person and online learning.

0.5 I believe that students should use data to guide their own learning progress.

0.6 I believe that teachers who regularly use data to inform their teaching will be able to help their students more than those who don’t.

0.7 I believe that students need to gain experience with online collaboration on projects.

0.8 I believe that student learning is enriched when teachers and students interact in online discussions.

1. Online Integration (Rate your ability to do the following . . . )

1.1 Evaluate the strengths and limitations of specific online activities for your students.

1.2 Strategically combine online and in-person activities that enable student ownership of their learning (e.g., flexibility in when, where, and how they learn).

1.3 Determine when it is best to use computer-administered assessments.

1.4 Determine when it is most effective to interact with students online versus in-person.

1.5 Discern when the use of online technologies improves learning outcomes and when it doesn’t.

1.6 Develop procedures for the online submission and management of student-created products (e.g., projects, reports, assignments, etc.).

1.7 Help students manage their class related online accounts and passwords.

1.8 Provide clear procedures for transitioning between online and in-person learning activities.

1.9 Establish procedures for how students should seek help when learning with online technology.

1.10 Establish guidelines that help students use online time wisely.

1.11 Establish clear procedures to help students manage the use of individual and/or classroom devices (laptops, tablets, headphones, etc.).

2. Real-time Data Practices (Rate your ability to do the following . . . )

2.1 Identify patterns in small group and whole-class learning based on online and offline assessment data.

2.2 Help students guide their own learning progress using online and offline assessment data.

2.3 Interpret mastery-based progress dashboards to inform interventions for individuals and small groups.

2.4 Use technology tools to monitor student participation levels (e.g., time on task, attendance, logins, frequency of activity, etc.).

2.5 Use frequent online formative assessments to monitor student progress towards learning outcomes.

2.6 Use online and offline assessment data to evaluate the effectiveness of accommodations for students with disabilities.

2.7 Use data collected online to determine which small groups or individual students need additional instructional support.

2.8 Use technology to collect non-academic data that can inform personalization of instruction (e.g., interests, learning preferences, background experiences, etc.).

3. Personalizing Instruction (Rate your ability to do the following . . . )

3.1 Organize a set of online and offline resources that support students in personalizing their projects.

3.2 Combine individual or small group instruction with the use of educational software to address specific student needs.

3.3 Use online tools to implement a mastery-based approach to learning.

3.4 Use educational software to customize student pathways through the curriculum.

3.5 Use technology that enables the pace of learning to be customized for each student.

3.6 Use technology to help students track progress on their personal learning goals.

3.7 Use technology to provide students with flexibility in where they learn.

3.8 Use technology to provide students with online options for how they demonstrate mastery of learning objectives.

4. Online Interaction (Rate your ability to do the following . . . )

4.1 Establish clear expectations for respectful online communication between students.

4.2 Establish clear boundaries when communicating online that maintain professional student-teacher relationships.

4.3 Facilitate productive small group collaborative project work that occurs both online and in-person.

4.4 Facilitate productive learner interaction in online discussion forums.

4.5 Facilitate productive video conferencing/webinar interactions with students and guest presenters.

4.6 Provide timely feedback to students using a variety of channels (text, audio, video, etc.).

4.7 Strengthen students’ sense of belonging to the classroom community using online communication.

4.8 Create opportunities for students to help each other inside and outside of class using online technology.

Keep up with the latest MVLRI has to offer