The Annotation Portal is a tool developed by Indiana University’s Institute for the Study of Contemporary Antisemitism (ISCA) and the Observatory on Social Media (OSoMe) to annotate potentially hateful social media posts. It can be used for many other types of annotation projects. Researchers who want to create an annotation project can upload content from X/Twitter and TikTok, as well as images and plain text. They can create annotation projects with a variety of question types (single and multiple choice, scaled, and annotated) and share their projects with their annotators. Social media posts from X/Twitter and TikTok are pulled into the portal. If an annotator wants to see the message in its original content, they can click on the post and be taken to the X/Twitter or TikTok page. There is no charge for using the annotation portal.

To create an annotation project, the first step is to create an account. You can then select "Create Annotation Project" and choose the type of content you want to work with. Once you have uploaded the appropriate file containing Tweet IDs, TikTok IDs, plain text, or images in the correct format, you will be able to create and edit questions for your annotation form. Once your form is complete, you can share the random token generated by the Annotation Portal with your annotators and/or use the token to do annotate the sample yourself. If you are an annotator and have received a token from your project manager, you should first create an account. Once logged in, you can select " Enroll in existing annotation project", enter your token, and start annotating.

To register, click on the "Register" link in the navigation menu. Provide a unique username, a valid email address, and a password that is at least 8 characters long and includes an uppercase letter. Once registered, you can log in with your credentials.

Use your registered username and password to log in. Click on the "Login" link in the navigation menu and enter your credentials. If you forget your password, you can initiate a password reset process from the login page.

Yes, once logged in, you can access your profile and make changes to your email address and password. Go to your profile page and find the respective options to update your email and password.

As a registered user, you can create a project by clicking on the "Create Annotation Project" link in the Annotation Project dropdown menu. Provide a unique name for the project, an optional description, and upload a docx or pdf file with instructions or rules for annotation.

After you create the project it will take you to upload a sample

As a registered user, you can create a sample after you create a project or if you already have a project created you can click on "View Samples" and then click on the button "Create a new sample".

You can choose from the following datasets: Twitter, Tiktok, Plain Text, or Images. Based on the selected dataset, a table will appear with specific information required in the CSV or JSON file for annotation.

As the project creator, you have project management capabilities. You can update the project type and information, add or edit mutliple questions for annotators, and modify the choices for the questions.

Project creator will receive an email notification with a unique token after the project is created. Then the annotator can access the project using this token, which grants them permission to participate in the annotation process.

Yes, annotators can save their progress by clicking "Next" after answering to atleast one question. This will submit their answer for that data and allow them to move on to the next one.

Yes, annotators can go back to what they annotate by click on the number display at the end of the project once they access the project and edit what they annotate.

Annotators have the option to export project information, including questions and their own answers, as a CSV file.

Yes, the project manager has access to export individual user answers as a csv or a zip file containing all annotator responses in seprate csv file.

Project managers can access project statistics, including basic metrics, and view a list of users enrolled in the project. They also have the ability to export user answers for individual users or the entire project as a zip file.