Tab periodicals Committee Periodicals Review and Advisory Committee (prac) Report for the ieee transactions on Industry Applications Date of Review

Demographic Summary

	2012 PRAC Report	November 2016
Total Number of AEs	147	135
Average AE Workload, papers	4.99	7.58
Organizational Diversity
Academia	51%	63.4%
Industrial	47.6%	36.6%
Government	1.4%	See note below
Regional Diversity (%) of AEs
1 through 6	57.9%	47.8%
7	8.3%	6.7%
8	20.7%	23.9%
9	1.4%	3.0%
10	11.7%	19.4%
Regional Diversity (%) of Membership
1 through 6	54%	47.9%
7	7.5%	5.6%
8	16.8%	17.8%
9	8.3%	8.8%
10	13%	19.9%
Gender Diversity (%)
Male	95.20%	97.0%
Female	4.80%	3.0%

This team of AEs reflects significant geographic and institutional diversity. It also includes significant representation from the major geographic constituencies in IAS. And in comparing the most recent statistics with those from our 2012 PRAC report, there is a noticeable shift toward a more global team.
Within IAS we have people who are employed by government regulatory agencies, government laboratories and various military organizations. There is a significant difference between working in a research laboratory and working on regulatory policy, and to generically group both sets under the heading ‘Government’ doesn’t make sense. Furthermore, many (and perhaps most) government-owned laboratories are actually managed by private industrial concerns. Hence, we have categorized AEs who work for government-owned laboratories as ‘industrial’, and AEs who work for government regulatory agencies as ‘government’
In our 2012 review, several members of the PRAC questioned the number of IAS Associate Editors. Our response was, and continues to be, that the very broad range of technical interests of the Society requires that we have Associate Editors who reflect the range of technologies covered by our constituents and the selection of topics addressed in the papers that come to us for review. As a consequence, we believe that our situation demands that we have more Associate Editors than would be the case in a Society that focuses on a narrower slice of technology.
However, we recognize the concern raised by the PRAC that the combination of a large number of Associate Editors and a large number of authors could suggest the possibility of collusion between authors and Associate Editors, eg, situations where an author of paper A is coincidentally the Associate Editor responsible for a different paper (B) whose list of authors includes the Associate Editor responsible for paper A, and the two individuals reach a quid-pro-quo agreement to assure that both papers are approved for publication. There is no evidence that this kind of improper behavior has ever actually happened in IAS. But to address the perception of a potential problem, we have reduced the total number of Associate Editors at the same time that we have experienced a significant increase in the number of submissions, effectively resulting in a significant increase in the workload imposed on individual AEs. AEs are volunteers. Obviously, this is not a trend that we will be able to maintain without limit, and we anticipate that if submissions continue to increase, we may have to cap workload and expand the number of AEs.

IAS takes a number of steps to regularly refresh the population of AEs supporting our publication activities. Because we use ScholarOne Manuscripts as our peer review management tool, we have total control over who can actually access the tool as an AE and therefore who can function in that capacity. Our practice is that access to S1M by all AEs expires on 1 February of even-numbered years – that forces the internal discipline of periodically assessing each AE and deciding if he or she will continue for another term. Obviously, because we are a volunteer-based organization, we always reflect the interests of individual volunteers, and attempt to accommodate the wishes of AEs who feel that they have done their share and wish to retire. Likewise, we take into account the wishes of our Technical Committees to introduce new people to the process. At the same time, we are sensitive to the very real issue of ‘title collectors’ who readily volunteer to become AEs so they can list that honor on their resume, but then are less enthusiastic about actually doing the work.

We use two hard metrics to objectively evaluate the performance of individual AEs– the number of papers that the AE has handled, and the average time to first decision reported on the papers handled by that AE. These parameters are calculated for each AE on a quarterly basis and the results are shared with the IAS Technical Committees whose activities they support, and when appropriate, we take hard decisions to retire underperforming AEs.
In addition, we periodically calculate overall performance data for our Society leadership that also can be used to determine how individuals compare with the total AE population. By looking at the entire picture, we can identify anomalies and determine if they represent problems that need to be addressed, or are rather just outliers in the overall performance of the Society. For example, the frequency distribution of average first decision time for each AE shown here discloses a few outliers – several are known situations involving AEs associated with Technical Committees whose traditions inevitably result in a longer time to complete the first decision, and also a temporary situation involving an otherwise well-performing AE whose attention was distracted by a job change during the period of measurement.
The frequency distribution of AE workload also shows wide variation. Because this reflects only a narrow window of time, there are naturally individuals in the population who were either ramping up, or phasing out, and therefore had fewer assignments during the measurement period.

3. Describe the process for Editor-in-Chief (EIC) selection and training, and policy on terms and reappointment.

The EiC is nominated by the Society Publications Department Chair and approved by the Society Executive Board in accordance with the Society Constitution and Bylaws. The person selected is expected to be familiar with the functioning of both the Society and the Transactions process (the four most recent EiCs have been past presidents of the Society). An effort is made to select an incoming EiC early enough so that both the incoming and outgoing EiCs may attend the annual IEEE Panel of Editors meeting together during the transition period. Further preparation includes close communication with the outgoing EiC both before and after the official date of change. The EiC also has available the guidance of the Society Publications Department Chair and the EiC of the IEEE Industry Applications Magazine and attends the annual IEEE Panel of Editors meetings during the EIC’s term of office.

The nominal term of office for the EiC is four years, with one renewal allowed, for a total of eight years. This is a recent development in IAS; some previous EiCs have served longer terms.

The outgoing Transactions EiC provides the incoming EiC with documents which describe activities and procedures. The current EiC periodically updates these documents in preparation for training the next EiC.

The Transactions Editor-in-Chief is assisted by a Manuscripts Administrator whose primary focus is to maintain the ScholarOne Manuscripts system, to assist users who are not familiar with that system, and to prepare reports from data stored on ScholarOne. The Manuscripts Administrator does not take an active role in managing individual papers, but rather serves as a facilitator to assist the EiCs and AEs in performing their functions.

4. Describe the process for Associate Editor selection and training, and policy on terms and reappointment.

In this context, it is important to understand that IAS is comprised of twenty (20) autonomous technical Committees with extremely divergent technical interests. For this reason, it is not possible for an Editor-in-Chief to have an understanding of the activities of every Committee. Therefore, our practice in IAS is to require that the Committees take a significant role in the peer review process. And as noted above, one AE in each Committee has specific responsibility to represent the IAS Publications process within the Committee organization (this person is usually called the Papers Review Chair for the Committee). Associate Editors are selected by the technical committees they serve with the approval of the EiCs of IAS Transactions and IAS Magazine, and typically are appointed for two year terms. One criterion for new AEs is experience as authors and/or reviewers. We do not specifically require that AEs be IEEE members, but to the best of our knowledge all of the AEs listed here are members of IEEE.

All AE appointments are set to expire on 1 February of even number years to impose the discipline of reviewing the performance of each person on a regular basis. Obviously, experience is a valuable characteristic for AEs, and there is a natural desire for high performing AEs to continue to serve the Society for a number of years. At the same time, it is important to weed out underperforming AEs, and also essential that AEs be allowed to retire voluntarily. An analysis is done of AE performance on a bimonthly basis that looks at both current workload and average manuscript turnaround time. Other factors include the experience of both the Technical Committee and the EiC is working with individual AEs. Based on these considerations, decisions are made about further service by each AE.
The distributed nature of our Society makes it impossible to get our AEs together at a conference or other meeting. Training is offered to new AEs (and to others who wish to refresh their understanding) in WebEx sessions typically held during November of each year. In addition, we have a set of documents that describe the duties and tasks involved in being an AE that we distribute periodically. Finally, our Manuscript Administrator publishes an FAQ e-mail approximately once each month to address recent problems, tips, policy matters, etc.

D. QUALITY
1. Describe handling of papers from submission to publication. Include a thorough description of the paper peer review process. For example, who reviews the first submission? How are papers distributed for review? To how many reviewers is each paper sent? Is there a summary review prepared by the editor? How many reviews are needed, at the minimum, to reach a decision?

Papers submitted to IAS describe innovations in a wide range of topics. Some examples are electric power protection and safety, electrostatics, power electronics, lighting and displays, automation and controls, and electric machines. The applications for these innovations span many industries, several of which are cement, pulp and paper, petrochemical, marine, alternate energy and transportation. For this reason, IAS Publications must work closely with the Technical Committees that make up the Society and who have the technical expertise necessary to competently conduct reviews of the wide range of papers that come before the Society for consideration. In 2008, IAS adopted ScholarOne Manuscripts (S1M) as its peer review management tool. Because the S1M site was designed by and is managed by IAS Publications, S1M is the mechanism by which the Publications Department enforces policies and uniform practices in peer reviews.

At the time of submission, and before papers go into review, there are two administrative checks performed on each paper. IAS requires that the IEEE Copyright Transfer be executed at the time of submission, and the first check is to confirm that the author has completed the transfer. If the transfer has not been executed, the author is sent a reminder, and the paper is flagged for a recheck before any decision is made for publication.

We also require that every submission be scanned in CrossCheck at the time of submission. The IEEE threshold is 30% similarity; in IAS, we scrutinize any paper that returns a similarity score in excess of 20%. Our experience is that the vast majority of instances of excessive similarity come from authors who are quoting their own prior work. The general approach we take is:

• 
  If the total similarity score is less than 20%, we don’t expose the paper to special scrutiny.
  
• 
  We require that any quotation of the authors own prior work that comprises more than 10% of the text of the paper must be identified by a reference citation, and if that citation is not present, the paper is returned to the author for correction.
  
• 
  We require that any quotation of work by others that comprises more than 10% of the text of the paper must be identified by a reference citation, and in addition, must be differentiated so that readers can clearly see that it is a quotation. Papers that fail to meet this requirement are returned to the author for correction.
  
• 
  If the similarity with any single source is large (typically > 60%), the paper is flagged and the AE is asked to work with the reviewers, as part of the review, to decide if the paper offers sufficient new content to justify publication.
  

The review process consists of five steps:

1. 
   An author of a paper presented at a conference sponsored or co-sponsored by IAS submits the paper for review (typically, within one year of the date of presentation). In 2016 there were 44 conferences sponsored or co-sponsored by IAS and over three thousand papers were eligible for review.
   
2. 
   Each of the twenty IAS Technical Committees has one or more Associate Editors, one of whom is designated as the Papers Review Chair for the Committee. This AE either takes the lead on managing the review of the paper, or else assigns the paper to another of the AE’s within the Committee. If that AE feels that the paper is outside the technical scope of the Committee, we have a process whereby responsibility can be transferred to a more suitable Committee in a way that is transparent to the author. If a suitable technical Committee cannot be found in IAS, and with the concurrence of the EIC, the author is notified that the paper is out of scope. (The role of the Papers Review Chair may require further explanation. The Papers Review Chair has been a part of the structure of most IAS Technical Committees for many decades. Typically the Papers Review Chair is an officer of the Committee and serves a two-year term. Before IAS adopted use of ScholarOne Manuscripts, each Papers Review Chair devised his or her own system to handle reviews. Currently, all Papers Review Chairs use ScholarOne to initiate and monitor the review process.)
   

3. 
   The AE selects and invites a minimum of two (a constraint that is imposed by ScholarOne Manuscripts) technical experts to review the paper. Reviewers are selected on the basis of their technical expertise relative to the subject of the paper.
   

• 
  AEs have the freedom to invite more than two reviewers, and frequently do so if they anticipate that having more than two formal reviews would be beneficial in reaching a decision.
  
• 
  There are three IAS technical committees with long-standing traditions of having a committee that reviews all papers coming to that committee for review. In those cases, all members of the review panel are asked to review each paper. The number of reviewers in these instances can range from the minimum of two to as many as nine.
  
• 
  AEs are repeatedly instructed, both in training sessions, in periodic FAQ e-mails and in written instructions, that it is a mandatory IEEE policy that reviewers be independent and that the Associate Editor cannot be a reviewer. Papers Review Chairs and AE’s have the obligation to recuse themselves in cases of possible conflict of interest. Some of the larger committees have designated one AE the specific duty of identifying and handling reviews for any situation where there is a question of bias.
  

4. 
   Reviewers download and read the paper, and then prepare a review. Each reviewer is asked to assess the paper against nine factors that were identified by IAS as indicators of the quality of a paper. These factors are presented in the form of multiple-choice questions to guide reviewers to think in terms of ‘to what degree’ rather than binary yes or no evaluations.
   

• 
  Does the paper match the technical interests of an IAS Technical Committee?
  
• 
  Does the paper make a significant contribution to technical understanding?
  
• 
  Does the paper contain information that should be archived for future reference?
  
• 
  Does the paper address new applications or technology?
  
• 
  Is the paper well written?
  
• 
  Is the paper concise?
  
• 
  Is the writing clear and understandable?
  
• 
  Do the figures and illustrations enhance the value of the paper?
  
• 
  Does the bibliography identify additional references on the subject of the paper?
  

In addition to answering these questions, reviewers are required to select one of four specific disposition recommendations for the paper:

• 
  Accept the paper for Transactions
  
• 
  Accept the paper for the Magazine
  
• 
  Return the paper to the author for revision and resubmission.
  
• 
  Reject the paper.
  

Finally, reviewers are asked to provide comments to the author whenever their recommendation is to either revise and resubmit or reject a paper. Reviewers are asked to differentiate between mandatory changes (changes that must be made for the reviewer to recommend the paper for publication), and suggested improvements. While authors are allowed to express a preference for publication in Transactions versus the Magazine, the actual decision is based on recommendations from reviewers based on criteria that are explained in the instructions provided to each reviewer. The actual decision is made by the EiCs of the Transactions and The Magazine. Those comments accompany the decision letter sent to the author.

5. 
   After the reviews are returned, the AE records a consensus decision that is communicated to the author.
   

It is sometimes the case that there is no natural consensus among reviewers. In that instance, there are several options available to the AE:

• 
  The most common scenario is where one reviewer recommends in favor of publication while a second recommends that the paper be rejected. An effective solution here is to report a ‘Revise and Resubmit’ decision to allow the authors to address the concerns identified by the reviewer who wanted to reject the paper. (Reviewers who recommend for rejection are required to explain their rationale.). Then, when the paper is resubmitted, if the authors have adequately addressed the negative reviewer’s concern, the paper can proceed to publication. Conversely, if the authors have not adequately addressed the concerns, the AE can be confident that rejection is the right disposition.
  
• 
  It is often possible for an astute AE to weigh the comments from the reviewers based on his own technical knowledge and background. Clearly, IEEE policies require that the AE be ‘independent’ (ie, not an active reviewer) and the reason is that in situations such as this, the AE may be called upon to be a ‘tie-breaker’.
  
• 
  Another option is for the AE to call for an additional reviewer. Unfortunately, this can have the adverse consequence of extending the time required to complete the review, so most AEs tend to choose one of the other two options listed above. However, this is often done in situations where authors challenge a decision to reject a paper.
  

The review process is single-blind: the AE and reviewers do see the list of authors, but the authors do not know the identity of the reviewers.

One of the features of the ScholarOne Manuscripts system is automatic tracking of reviewer-initiated revisions. As a result, we have seen a significant increase in the both the number of papers that receive a ‘revise and resubmit’ decision, and the number of revision cycles that individual papers go through. Historically, one of the problems with forcing authors to make decisions was that there was no control over the timeliness of those revisions. In implementing ScholarOne Manuscripts, we have imposed an automatic 30-day deadline for resubmission by authors, but AEs are also free to grant extensions upon request from author.

The decision to publish a paper in the Magazine rather than Transactions is made in the best interest of the members of the Society based on the nature of the paper. On rare occasions, an author may object to the decision to assign a paper to the Magazine or to the Transactions. Because the Magazine typically receives more publication recommendations than can be accommodated in a calendar year, these instances are almost always resolved to the satisfaction of the author when the Magazine EiC develops the final annual publication plan (discussed under Magazine).

While the detailed peer review is conducted by the AE and reviewers, the EiC retains final authority to accept or reject papers. As a practical matter, the EiCs rely on the AE and reviewers for their technical expertise, and the only time they are overruled is in the case procedural errors. In recent years, EiCs have overruled reviewer recommendations in cases where the author has not executed the IEEE copyright transfer, has been unable to provide permission to reprint copyrighted elements (figures) included in a paper, or has violated one or more IEEE or IAS policies.

Automation within S1M prevents authors from being assigned any role in connection with the review of their own paper or having any access to review records. If the Transactions EiC is an author, IAS practice is to delegate all decision-making responsibility on that paper to either the Magazine EiC, or the Chair of the IAS Publications Department.

Implementing S1M imposed a significant culture change on IAS. Prior to S1M, the process used for peer review was something that had evolved over many decades (the IAS Power Systems Engineering Committee was established in 1908 as the AIEE Committee on General Power Applications, the Industrial Lighting and Display Committee was formed as the AIEE Electric Lighting Committee in 1909, the Metal Industry Committee was created in 1914, and the Mining Industry Committee traces its heritage to about 1920, and there are records of activities by precursors of the Industrial Power Converter, Electric Machines, Rural Electric Power, Cement, Petroleum and Chemical Industry, and Pulp and Paper Industry Committees in AIEE prior to 1950.), and was essentially a ‘batch process’ with the beginning and ending points defined by specific calendar events (typically, regular IAS meetings). S1M is a true continuous process in which manuscripts and be submitted at any time, and where the completion time is governed by the progress on the review and not by external calendar events.

IAS has adopted an internal policy on the timeliness of peer review that focuses on getting to the first decision. The rationale is that if the disciplines and procedures are in place to deliver a timely first decision, then the final decision will follow in a proportionately timely fashion. The IAS policy contains two stipulations:

• 
  On papers that are submitted for review after the ‘presentation first’ requirement has been satisfied, the first decision should be reported no later than 90 days after submission.
  
• 
  On papers for which the ‘presentation first’ requirement is to be met after the paper has been submitted for formal peer review, the first decision should be reported no later than two calendar weeks after the close of the conference which IAS is a sponsor or co-sponsor at which the paper is presented.
  

Realistically, however, there will always be the occasional anomaly. So in addition to the routine reminders built into S1M, the IAS S1M site sends regular reports to the IAS Technical Committees that lists undecided papers. A report is distributed to IAS Executive Officers on a monthly basis that lists papers that have been in review for more than 90 days without a decision. The intention behind this report is to exert downward pressure within the organization to address review anomalies. Finally, a comprehensive set of performance reports is generated prior to each Society Executive Board meeting.

After the review process is concluded the author submits the final files for the paper. The EIC reviews the final files to verify that the files are complete and comply with IAS and IEEE policies. Papers which are missing information such as authors’ photos or a reference to the conference paper are put on hold until the author makes the necessary corrections. In most cases the issues are resolved within a few weeks.

Prior to 2012, the Transactions EiC held papers in S1M until just before the closing date of each issue, at which time they were exported as a batch to the Transactions Staff Editor for publication. Because there was a backlog of papers waiting for publication in the printed version of Transactions, this practice resulted in a long submission-to-publication time for individual papers. In 2012, the Transactions EiC started processing papers approved for publication in real time, exporting them to the Staff Editor immediately after the author submitted the final files required for publication. This change made it possible for the Staff Editor to post a preprint of each paper in Xplore within a few days of when the paper became available for publication. The result was a significant reduction in the average submission-to-ePublication time.