Chapter 3 Assessments Data/

3.1 Raw/

3.1.1 all_assignments.csv

This raw file is named all_assignments_[DATE].csv. File is generated by the AP-CAT system. It is NOT comma separated. In R, you will need to speficy separator is “\t”. File contains the following columns:

  1. assessment_group_id: unique number identifying class to which the assessment was administered.
  2. assessment_group_name: name of the class taking the assessment, either chosen by the teacher or the AP-CAT team. May not be unique, that is, for different pilot years of the study, there may exist classes that were given the same name. For example, “Period 1”.
  3. creator_id: ID number of the creator account who added the assessment into the system; assessments are added to a teacher’s class by a staff member of the AP-CAT project or by the teacher responsible for that class (in the case of practice assigments).
  4. due_date: date (and time) the assessment was due (set by the teacher responsible for the class).
  5. is_adaptive: whether the assessment was adaptive. If True, after the first 3 items, the next item was chosen based on the test-taker’s performance up until this point.
  6. is_published: status of the assessment at the time the data was retrieved.
    • If True, the assessment was published, and therefore it administered to students at least once.
    • If False, assessment was not published at the data was retrieved, that is, at that time, the assignment was not visible to any students.
    • If NA, the assessment was not published, and it was not administered to any students. This happens, for example, when an assignment is created on a staff’s account before being copied into an account of a teacher participating in the study.
  7. name: name given to the assessment at the time of administration.

3.1.2 IRT_calibration.csv

  1. Description:

    The name of this file is very misleading. It does NOT contain any type of item parameter calibration (that is, estimates of item difficulty). Instead, it contains the information you would use to obtain item parameters, i.e., the item responses.

    This is a long-format data file of item responses, blank-field answers, answer choices, and item-level response times. This data set contains one observation per row; each row corresponds to an answer given by a student to a question while taking a specific assignment. The same student might have answered the same question more than once (on assignments given at different times).

  2. Data columns:

    1. user_id: number that identifies each student account in the AP-CAT system. Student accounts might belong to high school students participating in the study, or they might belong to teachers or staff members that have created student “test accounts”.
    2. Qid: unique identifier for each question in the AP-CAT system. Renamed qid in other files.
    3. assignment_name: name given to the assessment in the AP-CAT system at the time of administration (see details in documentation.pdf)
    4. assessment_group_name: name of the class taking the assessment, either chosen by the teacher or the AP-CAT team. May not be unique, that is, for different pilot years of the study, there may exist classes that were given the same name. For example, “Period 1”.
    5. teacher_id: ID number of the teacher (creator) account who added the assessment into the system. Assessments may be added to a teacher’s class by a staff member of the AP-CAT project or by the teacher responsible for that class (in the case of practice assigments).
    6. is_correct: logical type of data indicating whether the item was answered correctly (“TRUE”) or incorrectly (“FALSE”), or NA if not shown to the student (if the question was given to student, but they skipped it, the question is automatically scored as incorrect).
    7. is_skipped: logical type of data indicating whether the student skipped the question.
    8. response_time: amount of time (in seconds) spent by the student on that item. If student moved on but then returned to question during the assessment, previous and current time are summed up.
    9. choice_id: unique number identifying which answer choice was selected by the student on that item. In order to make sense of these numbers, we need another file to match them to the answer choices shown for each question.
    10. blankfield_ans: numerical variable containing the observed answer for a blankfield item. If NA, item type is multiple-choice and not blankfield.

3.2 Clean/

3.2.1 Item Response & Time R script

Script to create master file named item-resp-time-data.csv containing item and time response data.

  1. Inputs

  2. Outputs

3.2.2 Item Response & Time master file

  1. Description:

    This file is named item-resp-time-data.csv and it contains both item responses and response times at the item level for each student, each item and each assessment administered by the AP-CAT projecs. This file was created with the script in resp-time-data.R and using the raw data file all_assignments_[DATE].csv.

    Note: data generated by practice or test assessments have been removed (see assessment_type variable description).

  2. Data columns:

    • user_id: unique number that identifies each student account in the AP-CAT system.
    • qid: unique identifier for each question in the AP-CAT system.
    • item_type: CREATE THIS VARIABLE. Item type is blankfield or multchoice.
    • multchoice_ans: CREATE THIS VARIABLE. answer choice was A, B, C or D, etc.
    • response_time: amount of time (in seconds) spent by the student on that item. If student moved on but then returned to question during the assessment, previous and current time are summed up.
    • is_correct: item response data indicating whether the item was answered correctly (1) or incorrectly (0), or NA if not shown to the student (if the question was given to student, but they skipped it, the question is automatically scored as incorrect).
    • is_skipped: indicates if student skipped (1) or not (0) a question that was given, or NA is question was not given to student in that assignment.
    • assessment_name: name given to the assessment in the AP-CAT system at the time of administration (see details in documentation.pdf)
    • assessment_group_name: name of the class taking the assessment, either chosen by the teacher or the AP-CAT team. May not be unique, that is, for different pilot years of the study, there may exist classes that were given the same name. For example, “Period 1”.
    • teacher_id: teacher ID responsible for the class taking the assessment
    • creator_role: actually don’t need this column but it is left over from the original dataset.