EARLI 2015

 

Review Process

Review Process

The review process is the most important aspect of organizing the EARLI conference, as the outcome of this process will determine the quality of the participants' experience.  As a result, EARLI strives to ensure that the process is rigorous and transparent.  In this note, we summarize the review process with the goal of a) informing EARLI members and conference participants about the review process, and b) reminding reviewers of the criteria used for the review process.

Please read on to learn about the:

1)       Selection process of the EARLI 2015 reviewers

2)      Review riteria

3)       Review notifications

 

1) SELECTION OF THE EARLI 2015 REVIEWERS 

Each of the 27 EARLI SIGs were asked to nominate reviewers for the conference; all reviewers need to be EARLI members of good standing, have a Ph.D. degree, and work at a research-intensive university.  These reviewers were, subsequently, confirmed by the conference organizers and affirmed by the EARLI Executive Committee, who approves the reviewer corpus.  Each of the reviewers was asked to indicate the EARLI SIG and EARLI domains within their expertise, which are then used for the automatic assignment of proposals by the EARLI conference submission system. In addition, proposals are assigned to reviewers who are not affiliated to institutions in the same country as any of the authors.

 

2) REVIEW CRITERIA

Each proposal is blindly reviewed by, at least, two referees, who are asked to take into account the criteria sohwn below, according to the type of contribution (empirical or theoretical, paper, poster, symposium, ICT demonstration, round table, workshop).   Symposia are evaluated both as a whole and as individual contributions.    

Each reviewer is asked to score the quality of each proposal, indicating a score of 1-10 for each dimension, and to provide brief but constructive feedback in a text box.  Whenever there is a total score discrepancy of 25 points or more, the proposal is automatically sent to a third reviewer.  The reviewers’ scores are averaged and taken into account when deciding if a proposal is accepted or not (see next section for more information about this).

Scores and comments are both communicated to the authors along with the final decision; we ask our reviewers to be as constructive as possible, while also being considerate of the efforts invested in writing a proposal.  

 

EMPIRICAL PAPER

  • Relevance to EARLI domain of Learning and Instruction
  • Significance for theory, policy and practice 
  • Theoretical framework, conceptual rationale or pragmatic grounding
  • Research method and design for both qualitative and quantitative approaches (research questions, context, participants, data sources, sampling, procedure, ethical issues)
  • Clarity of results or preliminary results and conclusions
  • Overall quality and scientific originality  

 Theoretical paper

  • Relevance to EARLI domain of Learning and Instruction
  • Significance for theoretical debate 
  • Theoretical framework, conceptual rationale or pragmatic grounding
  • Embeddedness in relevant literature
  • Clarity and robustness of theoretical argument
  • Overall quality and scientific originality

 POSTER (EMPIRICAL)

  • Relevance to EARLI domain of Learning and Instruction
  • Significance for theory, policy and practice 
  • Theoretical framework, conceptual rationale or pragmatic grounding
  • Research method and design (research questions, context, participants, data sources, sampling, procedure, ethical issues)
  • Preliminary data analysis or anticipated results
  • Overall quality and scientific originality  

POSTER (Theoretical)

  • Relevance to EARLI domain of Learning and Instruction
  • Significance for theoretical debate 
  • Theoretical framework, conceptual rationale or pragmatic grounding
  • Embeddedness in relevant literature
  • Clarity and robustness of theoretical argument
  • Overall quality and scientific originality

SYMPOSIUM (AS A WHOLE)

  • Relevance to EARLI domain of Learning and Instruction
  • Significance for theory, practice and policy
  • Theoretical perspective, conceptual rationale or pragmatic grounding
  • Organisation and internal logic of the whole symposium 
  • Overall quality and scientific originality

Each paper in a symposium will also be reviewed as an individual paper using the criteria outlined above.                                                                                                                                                                           

 ROUND TABLE

  • Relevance to EARLI domain of Learning and Instruction
  • Significance for theory, policy and practice 
  • Theoretical framework, conceptual rationale or pragmatic grounding
  • Research method and design (research questions, context, participants, data sources, sampling, procedure, ethical issues)
  • Clarity of issue at stake
  • Overall quality and scientific originality

ICT DEMONSTRATION

  • Relevance to EARLI domain of Learning and Instruction
  • Significance for theory, policy and practice 
  • Theoretical framework, conceptual rationale or pragmatic grounding
  • Validation in domain of application (as research method, data collection, research procedure, etc.)
  • Quality of workshop format (activity based) or demonstration activities
  • Overall quality and scientific originality

WORKSHOP

  • Relevance to EARLI domain of Learning and Instruction
  • Significance for theory, policy and practice 
  • Theoretical framework, conceptual rationale or pragmatic grounding
  • Validation in domain of application (as research method, data collecting, research procedure, etc.)
  • Quality of workshop format (activity based) or demonstration activities
  • Overall quality and scientific originality

3) REVIEW NOTIFICATIONS 

The following thresholds for accepting a proposal have been set by the EARLI 2015 International Scientific Committee and affirmed by the EARLI Executive Committee:

All scores refer to “combined review scores” (average of the review scores)

ScoreReview Decision
Less than 35

Automatic rejection for all contributions, including symposia.

Over 80

Automatic acceptance.

Between 36-79

Acceptance or rejection will be decided by the International Scientific Committee. Please note that the International Scientific Committee will not perform reviews but will make a decision based on, and respecting, the reviewers’ suggestions.

When a symposium is rejected the individual papers comprising the symposium are then evaluated on their own merit. Individual papers that are accepted based on the scores they have received are likely to be scheduled in the same paper session.  

Authors of proposals, whose reviews are complete and are within the automatic rejection or acceptance range, will automatically be notified by the system. This means that authors will be notified at different times, with all reviews expected to be communicated by the third week of March 2015.