This directory contains scripts and data for comparing with RankEF.

• organize_data.py: Processes the raw dataset files in data/ and organizes them into per-model directories with structured question and code files.

  • Usage:

    python comparison_with_rankef/organize_data_for_ranking.py

    Note: Run this script from the repository's root directory.

• pre_ranking_data.sh: Compresses or extracts the data necessary for RankEF's ranking process.

  • Usage:

    bash pre_ranking_data.sh

    Note: Run this script from inside THIS directory.

  • If the model directories exist and the archive does not, it creates pre_ranking_data.tar.xz.

  • If the archive exists and the directories do not, it extracts them.