rutaBAGA Walkthrough Tutorial

Overview

There are three main pages in the rutaBAGA system:

• Individual Rating (1): review candidates’ application documents, leave comments, and provide ratings.
• My Summary (2): summarize and interactively analyze your individual evaluations of candidates across selected criteria.
• Group Summary (3): summarize and interactively analyze aggregated group evaluations of all candidates and make final decisions.    

Individual Rating (1)

1. rutaBAGA allows application evaluation to be divided among the review team. Applicant reviewers can see the assigned applicants to review, assigned interviews, and other applicants.
2. Reviewers can navigate through application materials for the selected applicant by clicking any of the document links at the top of the window. Documents include personal statement (PS), resume, letters of recommendation (LOR), transcripts, and writing samples.
3. The profile window provides an overview of candidate background attributes and statistics. By clicking an element in the profile window, such as Name, the multi-selection drop down bar will appear. This bar allows users to select individual attributes which populate the profile section and, if desired, hide attributes that you do not want to bias your decision.
4. In the comments section, reviewers can add comments about the applicant, which can be viewed in later evaluation.
5. The ratings section allows the rating of applicant’s academic, research, program alignment, and Personal/Interpersonal Skills qualifications. While hovering on the respective criteria, a description of the attribute will appear. More information about rating is available below.

Additional information about rating applications:

Each prospect will be evaluated based on the following four criteria:

1. Academic Preparation: An applicant should demonstrate performance in CSI and other relevant STEM coursework that reflects an aptitude and readiness for success in CSI graduate courses and related academic processes. We review each applicant's academic achievements, including courses/credit load, academic honors, and traditional and non-traditional achievements.
2. Research and Scholarly Potential: An applicant should articulate aspirations and demonstrated ability or potential to conduct advanced, high-quality research in CSI-relevant domains. We look for evidence of the applicant's research or scholarly potential and enthusiasm for their chosen field of study through their research experiences, publications, presentations, or other achievements in their field. This may include awards or other recognition.
3. Program Alignment: An applicant should clearly articulate how the academic, scholarly, and research opportunities of our graduate programs align with their specific career and professional goals, passions, and experiences.
4. Personal/Interpersonal Skills: An applicant should demonstrate the ability to navigate successfully the personal and interpersonal challenges that abound in a PhD program, including:

• Communication Proficiency: Ability to communicate clearly in English, both verbally and written.
• Non-traditional Learning: We value applicants who have pursued diverse learning experiences beyond the traditional academic setting. This includes internships, work experience, entrepreneurial ventures, or any other impactful non-traditional learning encounters.
• Grit, Resilience, Empathy, and Collaboration: We consider these essential attributes for success in our program and beyond. Applicants who have demonstrated perseverance, resilience in the face of challenges, empathy in their interactions, and the ability to collaborate effectively with others are highly valued and will contribute to an inclusive and diverse academic community.
• Leadership or Community Involvement: We recognize the importance of leadership skills and community engagement. We seek applicants who have demonstrated leadership qualities, actively participated in academic and community initiatives, and shown a commitment to making a positive impact on society.

Each criterion will be evaluated on a 0-5 scale, as follows:

0: Clearly fails requirement

1: Likely fails requirement

2: Likely meets requirement

3: Clearly meets requirement

4: Likely exceeds requirement

5: Clearly exceeds requirement

Additionally, each applicant will be given an overall classification:

• Very Highly Competitive: exceeds all or most admission criteria
• Highly Competitive: exceeds some admission criteria (must meet all)
• Competitive: meets all admission criteria
• Not Competitive: fails to meet all or most admission criteria

My Summary (2)

1. Users can navigate between Home, Rating, My Summary, and Group Summary pages by selecting the desired tab. Group Summary will be disabled until all individual ratings are completed.
2. The Numerical Attributes window allows filtering on the interactive scatter plot (4.2) by numerical attributes such as GRE score, GPA, etc., as well as numerical ratings you have assigned for criteria such as Research Preparedness, Teaching Preparedness, etc.  
3. The Categorical Attributes window allows filtering on the interactive scatter plot based on overall classification and gender.
4. Navigation at the top allows you to go between the Time Spent Boxplots (4.1) and the Interactive Scatterplot (4.2).

1. The box plot data visualization reflects the time spent analyzing applications, stratified by gender and overall classification.  Candidates are represented as dots on the box plot. Hovering on a dot  populates the candidate’s profile, comments, and ratings. Clicking on a dot leads back to that candidate’s application portfolio in the Individual Ratings view.

2. The Interactive Scatterplot allows users to compare all the applicants they have rated. Users can select different application criteria or ratings for the X- and Y-axes to facilitate comparison. Points can be color-encoded based on applicants’ gender or overall classification and size-encoded to represent overall time spent on each applicant. Clicking on a dot leads back to that candidate’s application portfolio in the Individual Ratings view.

5. Beneath the scatterplot is a bar chart visualizing the amount of time spent focusing on individual application components. When hovering on a point in the scatterplot, dark blue bars are added to the bar chart to show how the distribution of time for an individual applicant compares to the reviewer’s average (see image below).

Group Summary (3)

The Group Summary view is available once all independent ratings have been completed.

1. The Numerical Attributes window allows filtering data displayed on the interactive scatter plot by adjusting the sliders.  
2. The Categorical Attributes allows filtering on the interactive scatter plot based on admission decision, average overall classification and gender.
3. Navigation at the top allows you to go between the Time Spent Boxplots (3.1) and the Interactive Scatterplot (3.2).

1. The box plot data visualization reflects the average time reviewers spent analyzing applications, stratified by gender and overall classification. Reviewers are represented as dots on the box plot. Clicking on a dot will highlight the reviewer (indicated by thicker strokes) in all the boxes .

2. The Interactive Scatterplot allows users to compare all the applicants they have rated. Users can select different application criteria or ratings for the X- and Y-axes to facilitate comparison. Points can be color-encoded based on applicants’ gender, average overall classification, or admission decision and size-encoded to represent average overall time spent on each applicant. Clicking on a dot populates the candidate’s profile, comments, and ratings. Right clicking leads back to that candidate’s application portfolio in the Individual Ratings view.

4. The profile window provides an overview of candidate background attributes and statistics. Beneath the profile information is a table showing the reviewers who rated the given applicant, including the rating they applied, time spent reviewing this applicant, and their averages for these values across all applications they reviewed in parentheses. Clicking on the info icon in the top left of this view will lead back to the applicant’s portfolio in the Ratings view.
5. In the comments section, reviewers can view aggregated comments about the applicant from all reviewers.
6. The ratings section shows the aggregated ratings of an applicant’s academic, research, teaching, and communication qualifications in a strip plot.
7. The decision panel initially places all applicants in the Undecided list. Applicants can be sorted by last name, average overall classification, or total number of reviewers by selecting the arrows in the header of the list. After discussion, applicants can be moved to the appropriate list reflecting their admissions decision: Admit, Waitlist, or Reject. Only committee chairs can move applicants to different decision lists.