Galaxy Zoo

Galaxy Zoo#

Overview#

In Project 01 you explored the Galaxy Zoo 2 decision tree and the vote fractions that volunteers produced for 61,578 SDSS galaxies. In this project you will train a convolutional neural network (CNN) to predict those vote fractions directly from the galaxy images, as in the Kaggle Galaxy Challenge [5] and its winning solution [2].

Data Sources#

File (course mirror)	Size	Contents
`training_solutions_rev1.zip`	4.9 MB	The labels: `GalaxyID` + 37 vote fractions for each of the 61,578 training galaxies
`images_training_rev1.zip`	830 MB	61,578 training images, `<GalaxyID>.jpg`, 424 × 424 pixel RGB (SDSS gri colour composites)
`images_test_rev1.zip`	1.1 GB	79,975 test images. The competition’s evaluation set; no labels are provided
`all_zeros_benchmark.zip`, `all_ones_benchmark.zip`, `central_pixel_benchmark.zip`	< 1 MB each	Example submissions (predictions for the test galaxies) from trivial baseline methods. They show the submission format; they are not labels

The original data are on Kaggle; downloading them there requires a Kaggle account and accepting the competition rules.

About the labels. The 37 numbers for each galaxy are not independent class probabilities. They are the weighted fractions of Galaxy Zoo volunteers who gave each response, propagated down the decision tree. The answers to the first question (Class 1) sum to 1. The answers to each later question sum to the fraction of volunteers who reached that question. For example, Class 2 (“edge-on?”) sums to Class 1.2 (“features or disk”).

Important:

Train on images_training_rev1 with the targets from training_solutions_rev1.csv. The *_benchmark files are trivial example predictions, not labels.
The labels of the test images were never released publicly. Make your own training, validation and test split from the 61,578 labelled galaxies.
Full-resolution images take a lot of memory: 61,578 × 424 × 424 × 3 bytes ≈ 33 GB. Most of each image is empty sky, so crop the central region and downsample it. For example, Ref. [2] (Section 7.3) cropped to 207 × 207 and then downscaled by a factor of 3, to 69 × 69. The helper below does this and stores the images as uint8. At 64 × 64 the full set takes about 760 MB.

import io
import os
import urllib.request
import zipfile

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from PIL import Image
from sklearn.model_selection import train_test_split

BASE_URL = "https://courses.physics.illinois.edu/phys498mlp/sp2025/data/projects/GalaxyZoo/"

def fetch(filename):
    """Download a file from the course mirror once; later calls reuse the local copy."""
    if not os.path.exists(filename):
        urllib.request.urlretrieve(BASE_URL + filename, filename)
    return filename

labels = pd.read_csv(fetch("training_solutions_rev1.zip"))
train_zip = zipfile.ZipFile(fetch("images_training_rev1.zip"))  # ~830 MB, downloaded once

def load_images(galaxy_ids, crop=212, size=64):
    """Center-crop each 424x424 image to `crop` pixels, resize to `size`, and return a uint8 array (N, size, size, 3)."""
    lo = (424 - crop) // 2
    out = np.empty((len(galaxy_ids), size, size, 3), dtype=np.uint8)
    for i, gid in enumerate(galaxy_ids):
        img = Image.open(io.BytesIO(train_zip.read(f"images_training_rev1/{gid}.jpg")))
        out[i] = np.asarray(img.crop((lo, lo, lo + crop, lo + crop)).resize((size, size), Image.BILINEAR))
    return out

N_GALAXIES = 10_000  # start small; use all 61,578 for your final results (loading takes a few minutes)
subset = labels.sample(n=N_GALAXIES, random_state=42) if N_GALAXIES < len(labels) else labels
X = load_images(subset["GalaxyID"].to_numpy())
Y = subset.drop(columns="GalaxyID").to_numpy(dtype=np.float32)  # 37 targets per galaxy

# 70% train / 15% validation / 15% test
X_train, X_tmp, Y_train, Y_tmp = train_test_split(X, Y, test_size=0.3, random_state=42)
X_val, X_test, Y_val, Y_test = train_test_split(X_tmp, Y_tmp, test_size=0.5, random_state=42)
print(X_train.shape, X_val.shape, X_test.shape, Y_train.shape)

Questions#

General instructions

Use PyTorch, as in the Convolutional and Recurrent Neural Networks, Deep Learning and GPU 101 lectures.
Keep the test set untouched until your final evaluation. Use the validation set for choosing hyperparameters and for early stopping.
Plot training and validation loss versus epoch for every network you train.
Report the number of trainable parameters, the training time, and the hardware used.

Question 01#

A simple CNN. Design, describe and implement a simple CNN that takes a galaxy image as input and predicts the 37 vote fractions.

What exactly is the input to your network (crop, resolution, colour channels, normalization)? Justify your choices.
What is the output layer, and which loss function do you use? The targets are fractions between 0 and 1, so think about an appropriate output activation.
Report the RMSE over all 37 outputs on your test set (the Kaggle metric). Compare it with the trivial baseline of predicting the training-set mean label for every galaxy (see the optional extension of Project 01). For reference, the winning Kaggle entry [2] reached an RMSE of about 0.075 on the competition’s test set. Your test set is different, so the numbers are not directly comparable.

Question 02#

How well does it classify? Turn your predictions into classifications for the first question of the decision tree by taking the answer (smooth, features/disk, or star/artifact) with the largest predicted Class 1 value. Compare with the volunteers’ majority answer.

Show the confusion matrix and the accuracy on your test set.
Plot predicted versus true vote fractions for a few outputs (e.g. Class 1.1, Class 2.1, Class 4.1).
Show some of the galaxies your network gets most wrong. Can you see why?

Optional extensions

Galaxy images have no preferred orientation. Add data augmentation (random rotations, flips, small shifts and zooms) as in Section 7.4 of Ref. [2], and measure the effect on your RMSE.
The 37 outputs must obey the constraints of the decision tree (Project 01, Question 05). Implement an output layer that enforces them, as described in Section 7.6 of Ref. [2], and compare it with an unconstrained output.

References#

[1] K. W. Willett et al., “Galaxy Zoo 2: detailed morphological classifications for 304,122 galaxies from the Sloan Digital Sky Survey,” Mon. Not. R. Astron. Soc. 435, 2835 (2013). doi:10.1093/mnras/stt1458, arXiv:1308.3496

[2] S. Dieleman, K. W. Willett, J. Dambre, “Rotation-invariant convolutional neural networks for galaxy morphology prediction,” Mon. Not. R. Astron. Soc. 450, 1441 (2015). doi:10.1093/mnras/stv632, arXiv:1503.07077

[3] C. Lintott et al., “Galaxy Zoo 1: data release of morphological classifications for nearly 900,000 galaxies,” Mon. Not. R. Astron. Soc. 410, 166 (2011). doi:10.1111/j.1365-2966.2010.17432.x, arXiv:1007.3265

[4] D. G. York et al., “The Sloan Digital Sky Survey: Technical Summary,” Astron. J. 120, 1579 (2000). doi:10.1086/301513, arXiv:astro-ph/0006396

[5] Galaxy Zoo – The Galaxy Challenge, Kaggle competition (2013–2014). https://www.kaggle.com/competitions/galaxy-zoo-the-galaxy-challenge

Acknowledgements#

Initial version: Mark Neubauer
Revised: 2026