ManyTypes4Py: A Benchmark Python Dataset for Machine Learning-based Type Inference
Paper • 2104.04706 • Published
Dataset Viewer
The dataset viewer is not available for this dataset.
Unexpected token '<', "<html>
<h"... is not valid JSON
Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.
ManyTypes4Py-Reconstructed
This is a reconstruction of the original code from the ManyTypes4Py paper from the following paper
A. M. Mir, E. Latoškinas and G. Gousios, "ManyTypes4Py: A Benchmark Python Dataset for Machine Learning-based Type Inference," IEEE/ACM International Conference on Mining Software Repositories (MSR), 2021, pp. 585-589
The artifact (v0.7) for ManyTypes4Py does not have the original Python files. Instead, each file is pre-processed into a stream of types without comments, and the contents of each repository are stored in a single JSON file. This reconstructed dataset has raw Python code.
More specifically:
We extract the list of repositories from the "clean" subset of ManyTypes4Py, which are the repositories that type-check with mypy.
We attempt to download all repositories, but only succeed in fetching 4,663 (out of ~5.2K).
We augment each file with the text of each type annotation, as well as their start and end positions (in bytes) in the code.
Internal Note
The dataset construction code is on the Discovery cluster at /work/arjunguha-research-group/arjun/projects/ManyTypesForPy_reconstruction.
- Downloads last month
- 128