The How2Sign dataset consists of a parallel corpus of speech and transcriptions of instructional videos and their corresponding American Sign Language (ASL) translation videos and annotations. It has been produced by recording 11 persons (6 males and 5 females) with various hearing status (5 self-identified as hearing, 4 as deaf, 2 as hard of hearing). During the study the upper body was recorded (face, mouth, arms and Hands). The video has been recorded at 30 fps in MPEG format. A total of 80 hours of Multiview American Sign Language videos were collected, as well as gloss annotations and a coarse video categorization. The data is available directly at the following link: https://how2sign.github.io/#download