Review of Python's Role in VisIt

[aashish24]

Independent Review Report

Reviewer: Aashish Chaudhary

Department/Center/Division: Scientific Visualization

Institution/University/Company: Kitware Inc.

Field of interest / expertise: Scientific Computing / Visualization

Country: USA

Article reviewed:

GENERAL EVALUATION

Please rate the paper using the following criteria (please use the abbreviation
to the right of the description)::

below doesn't meet standards for academic publication
meets meets or exceeds the standards for academic publication
n/a not applicable

• Quality of the approach:
  meets
• Quality of the writing:
  below. I have given the quality of writing below because the author(s) have not described the challenges in using / implementing various features in Python very clearly upfront. Also, comparison with other similar toolkits could have used some more detail.
• Quality of the figures/tables:
  meets

SPECIFIC EVALUATION

For the following questions, please respond with 'yes' or 'no'. If you
answer 'no', please provide a brief, one- to two-sentence explanation.

• Is the code made publicly available and does the article sufficiently
  describe how to access it? We aim not to publish papers that essentially
  advertise proprietary software. Therefore, if the code is not publicly
  available, please provide a one- to two- sentence response to each of the
  following questions:
  yes the code is publicly available.
  • Does the article focus on a topic other than the features
    of the software itself?
    No. The article mostly focuses on use of Python in Visit.
  • Can the majority of statements made be externally validated
    (i.e., without the use of the software)?
    No. Since the features described are not mathematical theories or proofs.
  • Is the information presented of interest to readers other than
    those at whom the software is aimed?
    Yes. I think the paper describes the use of Python really well in a parallel computing
    environment.
  • Is there any other aspect of the article that would
    justify including it despite the fact that the code
    isn't available?
    N/A
  • Does the article discuss the reasons the software is closed?
    Software is open source
• Does the article present the problem in an appropriate context?
  Specifically, does it:
  • explain why the problem is important,
    Somewhat. Would have been nice to state problem and challenges more clearly.
  • describe in which situations it arises,
    yes
  • outline relevant previous work,
    Somewhat.
  • provide background information for non-experts
    Not sufficient.
• Is the content of the paper accessible to a computational scientist
  with no specific knowledge in the given field?
  Yes
• Does the paper describe a well-formulated scientific or technical
  achievement?
  Yes (technical achievement)
• Are the technical and scientific decisions well-motivated and
  clearly explained?
  Somewhat. It would have been nice to outline reasons to pick python.
• Are the code examples (if any) sound, clear, and well-written?
  Yes
• Is the paper factual correct?
  Yes
• Is the language and grammar of sufficient quality?
• it could be improved a bit. At some places I found missing comma separation.
• Are the conclusions justified?
  Yes
• Is prior work properly and fully cited?
  Somewhat. It would have been nice if author(s) cited some more related work
  and compare it with their own bit more in detail.
• Should any part of the article be shortened or expanded? Please explain.
  I think length of the paper is good.
• In your view, is the paper fit for publication in the conference proceedings?
  Please suggest specific improvements and indicate whether you think the
  article needs a significant rewrite (rather than a minor revision).
  The paper presented the use of Python in various forms in VisIT for scientific computing and visualization. I believe implementing all the features described in this paper within python is difficult and I think author(s) did a great job implementing these in the toolkit. Since this is a technical paper, it would have been nice to compare the work performed with prior work with respect to:
• Performance: Evaluate whether or not performance have been improved compare to previous implementations.
• Ease of use: Did the API and use of Python made it easier to use VisIT in HPC environment?
  If yes, a user study would have been nice to confirm it.

Overall, I am pleased with the work described here and I am giving it a "may be" for the acceptance.

[wfspotz]

Independent Review Report

Reviewer: Bill Spotz

Department/Center/Division: Computing Research

Institution/University/Company: Sandia National Laboratories

Field of interest / expertise: Scientific computing

Country: USA

Article reviewed:

GENERAL EVALUATION

Please rate the paper using the following criteria (please use the abbreviation
to the right of the description)::

below doesn't meet standards for academic publication
meets meets or exceeds the standards for academic publication
n/a not applicable

• Quality of the approach: meets
• Quality of the writing: below (The manuscript could use another iteration of editing for grammar. Also the VisIt Python interface is quite non-pythonic and modal. There should be some discussion about why this is the case.)
• Quality of the figures/tables: meets

SPECIFIC EVALUATION

For the following questions, please respond with 'yes' or 'no'. If you
answer 'no', please provide a brief, one- to two-sentence explanation.

• Is the code made publicly available and does the article sufficiently
  describe how to access it? We aim not to publish papers that essentially
  advertise proprietary software. Therefore, if the code is not publicly
  available, please provide a one- to two- sentence response to each of the
  following questions:
  Yes
  • Does the article focus on a topic other than the features
    of the software itself?
    No. This focus is appropriate for an article of this nature.
  • Can the majority of statements made be externally validated
    (i.e., without the use of the software)?
    No. Validation would require use of the software.
  • Is the information presented of interest to readers other than
    those at whom the software is aimed?
    Yes.
  • Is there any other aspect of the article that would
    justify including it despite the fact that the code
    isn't available?
    N/A
  • Does the article discuss the reasons the software is closed?
    N/A
• Does the article present the problem in an appropriate context?
  Specifically, does it:
  • explain why the problem is important,
    Yes
  • describe in which situations it arises,
    Yes
  • outline relevant previous work,
    No. A discussion of other visualization software is absent.
  • provide background information for non-experts
    No. Visualization is sufficiently omnipresent that this paper is not really aimed at experts.
• Is the content of the paper accessible to a computational scientist
  with no specific knowledge in the given field?
  Yes
• Does the paper describe a well-formulated scientific or technical
  achievement?
  Yes
• Are the technical and scientific decisions well-motivated and
  clearly explained?
  No. As I mentioned before, the VisIt Python interface is modal, i.e. commands put the software in a variety of modes. This is not object oriented and is non-pythonic. Why this is the case is not explained or motivated.
• Are the code examples (if any) sound, clear, and well-written?
  Yes
• Is the paper factually correct?
  Yes
• Is the language and grammar of sufficient quality?
  No. It could use another iteration.
• Are the conclusions justified?
  Yes
• Is prior work properly and fully cited?
  Yes (libraries and tools used by VisIt are properly cited, but competing visualization tools are not mentioned)
• Should any part of the article be shortened or expanded? Please explain.
  Yes. I would like to see a discussion of the modal nature of the interface.
• In your view, is the paper fit for publication in the conference proceedings?
  Please suggest specific improvements and indicate whether you think the
  article needs a significant rewrite (rather than a minor revision).
  Yes, it is fit for publication, with a minor editing pass and the inclusion on a discussion of the interface design decisions.

[cyrush]

Hi Aashish & Bill,
Thanks for your reviews -- I worked to address the reviews with edits included in Pull Request #25
-Cyrus