Playground: Chapter 2

Data Structures in Python

This page practises the ideas of Chapter 2 in four parts. Practise the chapter works through the book’s exercises, with starter code, hints, and solutions. Go further goes beyond the book, with new questions and datasets. Check your understanding asks short conceptual questions whose answers open with a click. Do it yourself gives open tasks with an empty code space and no answers: you decide how to write the code, as you will in your own thesis.

In the exercises, replace each ______ with your own code and press Run Code.

Practise the chapter

These are the exercises at the end of Chapter 2, with the same numbers.

Exercise 1: Ages over 30

Create a Pandas Series with the ages 24, 31, 28, 45, and 26. Use boolean indexing to filter only the ages over 30, and count how many there are.

NoteHint

The condition ages > 30 generates a boolean Series of True and False. Inside square brackets it filters the values; calling .sum() counts how many values evaluate to True.

TipSolution 
import pandas as pd

ages = pd.Series([24, 31, 28, 45, 26])
print(ages[ages > 30])
print((ages > 30).sum())

Two ages (31 and 45) exceed 30. The same boolean expression filters the rows and counts qualifying cases.

Exercise 2: Ordered categories

Create an ordered categorical Series from ["Agree", "Disagree", "Neutral", "Agree"] whose categories run from "Disagree" through "Neutral" to "Agree". Check the frequency distribution with .value_counts().

NoteHint

Pass the ordered list ["Disagree", "Neutral", "Agree"] to the categories parameter. Use sort=False in .value_counts() to preserve the defined category order.

TipSolution 
import pandas as pd

answers = pd.Series(
    pd.Categorical(
        ["Agree", "Disagree", "Neutral", "Agree"],
        categories=["Disagree", "Neutral", "Agree"],
        ordered=True
    )
)
print(answers.value_counts(sort=False))

Setting ordered=True establishes an explicit scale order: Disagree (1), Neutral (1), and Agree (2).

Exercise 3: Part-time students with children

In the students DataFrame, count the number of students who study part-time (study_mode == "Part-time") AND have dependent children (children == "Yes").

NoteHint

In Pandas, combine multiple boolean conditions using the bitwise operator & (AND). Each condition must be wrapped in its own parentheses ().

TipSolution 
condition = (students["study_mode"] == "Part-time") & (students["children"] == "Yes")
print(condition.sum())

51 students in the cohort are part-time students with children.

Exercise 4: A new column

Add a new column over_30 to students that evaluates to True for students older than 30. Count how many students qualify, and check for missing values.

NoteHint

Use students["age"] > 30. .isna().sum() counts how many values in students["age"] are missing.

TipSolution 
students["over_30"] = students["age"] > 30
print("Students over 30:", students["over_30"].sum())
print("Missing age values:", students["age"].isna().sum())

205 students are over 30 years old, while 1 student has a missing age observation (NaN).

Exercise 5: The question behind a variable

When working with SPSS (.sav) files in Python, we extract question wording and survey metadata. Explain what a variable label provides that a raw column name like support_3 cannot convey alone.

A column identifier like support_3 is an internal shorthand. In the SPSS file metadata (meta.column_names_to_labels["support_3"]), the full survey item prompt is recorded: “My supervisor cares about my wellbeing.” Without the prompt text and answer anchors (1 = Strongly Disagree to 5 = Strongly Agree), quantitative values are uninterpretable.

Exercise 6: Units of analysis

State the unit of analysis of each table in the study (students, semesters, questionnaire, and supervisors). For each, name one research question it could answer on its own.

  • students: An individual graduate student (\(N = 600\)). Question: Are part-time students more likely to report intentions to drop out than full-time students?
  • semesters: A single semester observation for a student (\(N = 2,326\)). Question: Does average sleep duration decline from semester 1 to semester 4?
  • questionnaire: A student’s baseline psychological survey (\(N = 600\), 22 Likert items). Question: What is the correlation between perceived stress and academic burnout?
  • supervisors: A faculty supervisor (\(N = 120\)). Question: Does the number of supervised graduate students vary significantly by academic department?

Go further

These exercises explore more advanced data indexing and codebook generation.

Exercise 7: A DataFrame of mixed types

Create a DataFrame for five participants containing a nominal category, an ordered category, an integer age, and a boolean employment flag. Inspect the architecture using .info().

NoteHint

Call .info() on the DataFrame five to view column types and memory usage.

TipSolution 
import pandas as pd

five = pd.DataFrame({
    "faculty": pd.Categorical(["Science", "Education", "Science", "Engineering", "Education"]),
    "exercise": pd.Categorical(
        ["Rarely", "Often", "Sometimes", "Rarely", "Often"],
        categories=["Rarely", "Sometimes", "Often"],
        ordered=True
    ),
    "age": [26, 34, 29, 41, 25],
    "works": [True, False, False, True, True]
})
print(five.info())

The output confirms the four distinct data types: nominal category, ordered category, integer int64, and boolean bool.

Exercise 8: Finding participants with missing data

Using .loc[] and .isna(), show the student_id, faculty, and programme of every student whose age is missing.

NoteHint

Call students["age"].isna() inside the row index of .loc[].

TipSolution 
missing_students = students.loc[students["age"].isna(), ["student_id", "faculty", "programme"]]
print(missing_students)

Exactly one participant has a missing age: student S0007, a Master’s student in Health Sciences.

Exercise 9: Multi-wave panel records

Filter all rows for student S0001, count the total rows in semesters, count unique student IDs, and count observations per semester.

NoteHint

The method .nunique() counts the number of distinct unique values in a Pandas Series.

TipSolution 
print(semesters[semesters["student_id"] == "S0001"])
print("Total rows:", len(semesters))
print("Unique students:", semesters["student_id"].nunique())
print(semesters["semester"].value_counts())

Student S0001 has 4 rows. While there are 2,326 total semester observations, there are 600 unique students. 37 students did not complete all 4 semesters.

Exercise 10: Building an automated codebook

Construct a summary DataFrame documenting column names, descriptions, and Python dtypes for a study dataset.

TipSolution 
import pandas as pd

cols = ["student_id", "age", "faculty"]
meanings = ["Unique student code", "Age at baseline", "Academic faculty"]

codebook = pd.DataFrame({
    "variable": cols,
    "meaning": meanings,
    "dtype": [str(students[c].dtype) for c in cols]
})
print(codebook)

An automated codebook ensures clear documentation of study variables.

Check your understanding

Answer each question in your own words first, then click to see the model answer.

1. Why does pd.Series([6.5, "seven"]) become an object/string Series rather than remaining numeric?

A Pandas Series requires a homogeneous data type. Because text cannot be coerced into a float without data loss, Pandas coerces the entire Series to object (string) dtype, preventing accidental mathematical calculations on text.

2. What is the difference between df.iloc[0] and df.iloc[1]?

Python uses 0-based indexing: df.iloc[0] returns the first row (offset 0 from the start), while df.iloc[1] returns the second row (offset 1).

3. What is the advantage of using pd.Categorical(..., ordered=True) for ordinal Likert scales?

Setting ordered=True establishes an explicit hierarchy among categories. This ensures that frequency tables and figures display categories in logical order (e.g. Disagree < Neutral < Agree) rather than alphabetical order, and allows logical comparisons.

4. Why must an analysis of student demographics count each student once, even when the semester dataset contains multiple rows per student?

The unit of analysis for demographic questions is the person, not the semester. Counting rows in a multi-wave table weights students who stayed longer more heavily and violates the assumption of independent observations.

5. What vital information does an automated codebook provide that raw CSV column headers omit?

A codebook documents the semantic meaning: the exact survey question wording, the response scale anchors, the encoding of numeric codes, and conventions for missing observations.

Do it yourself

These open tasks have no starter code and no answers. Write your own Python code, as you would for your thesis.

1. Import the semester data twice: once from semesters.csv with pd.read_csv(), and once from the Excel export wellbeing_raw.xlsx with pd.read_excel(). Compare their .shape and column types.

2. Select three variables from students (e.g. gender, employment, financial_worry). Convert each into its appropriate measurement type (category, ordered category, or numeric) and justify your choices.

3. Load the built-in penguins dataset from seaborn (sns.load_dataset("penguins")). Identify its unit of analysis, count missing values across columns, and compute mean body mass by species.

4. In your local thesis project: write a script create_codebook.py that reads an SPSS or CSV file, compiles variable names, types, and descriptions into a DataFrame, and exports it to codebook.csv with index=False.

Work on your own computer

NoteDownload the Chapter 2 project

The starter project contains the data and all exercises as executable Python scripts: starter exercises with blanks, solutions, and open tasks.

  • Download chapter02.zip, unzip it, and open the folder in VS Code.
  • Run pip install data2thesis pandas pyreadstat openpyxl in your Python environment.