Pipeline Understanding Experiment

Research Question

“What reasons exist for effects on bottom-up program comprehension for data pipelines implemented in Jayvee compared to Python for non-professional programmers?”

Codebook

Category PL - Programming Language

• PL1 - Pipeline Overview
  • PL1.1 - Information is split between pipeline overview and implementation
  • PL1.2 - Without pipeline overview, all code must be considered
  • PL1.3 - Python does not provide a pipeline overview
  • PL1.4 - Jayvee pipeline overview improves speed
  • PL1.5 - Jayvee pipeline overview helps to understand the flow of a pipeline
• PL2 - Code Structure
  • PL2.1 - Sequential structure of Jayvee
  • PL2.2 - Blocks structure improves understanding
  • PL2.3 - Encapsulation of related code in blocks or functions improves understanding
  • PL2.4 - Jayvee enforces a structure
  • PL2.5 - No clearly defined or enforced structure in Python
• PL3 - Transparency
  • PL3.1 - Hidden implementation details make understanding difficult
  • PL3.2 - Automation decreases understanding
  • PL3.3 - Interpretation blocks make outcome unclear
  • PL3.4 - Scoped code (blocks, functions) should execute one operation, not multiple
  • PL3.5 - Density of functionality effects understanding
• PL4 - Amount of Options
  • PL4.1 - Amount of functionality Limited and General
  • PL4.2 - External libraries have different approaches and change
  • PL4.3 - Trade-off between customization and defaults
  • PL4.4 - Different functions to achieve the same goal decrease understanding
• PL5 - Syntax
  • PL5.1 - Syntax is important for novices
  • PL5.2 - Unfamiliar syntax makes understanding difficult
  • PL5.3 - Code can be read like english text
  • PL5.4 - Expressive and verbose syntax
  • PL5.5 - Different syntax options for the same semantic decrease understanding
• PL6 - Language Elements
  • PL6.1 - Advanced programming concepts like map are hard to understand
  • PL6.2 - Domain-specific language elements like blocks make Jayvee easy to understand
  • PL6.3 - Jayvee language elements (blocks, value types, constraints) are unusual and need to be learned
  • PL6.4 - Meta infos (e.g., imports, library names) obscure the logic of the pipeline
  • PL6.5 - Mixed effects of no variables in Jayvee

Category HU - Human Factors

• HU1 - Required Experience
  • HU1.1 - Python is harder to understand without previous experience
  • HU1.2 - Novices profit from good naming
  • HU1.3 - Good programming habits are required to keep Python easy to understand
  • HU1.4 - Less structure can lead to hard to understand implementation
• HU2 - Applicable Experience
  • HU2.1 - Mental model of an ETL pipeline aligns with Jayvee
  • HU2.2 - No learning effect between pipelines due to low structure
  • HU2.3 - Participants had previous experience with Python
  • HU2.4 - Participants had no experience with Jayvee and its language design
• HU3 - Naming
  • HU3.1 - Jayvee names describe functionality well
  • HU3.2 - Python function names are unclear or inconsistent
  • HU3.3 - Good names for steps is important to understanding

Category EX - External Elements

• EX1 - Documentation
  • EX1.1 - Uniform documentation structure improves speed
  • EX1.2 - Quality of documentation of Jayvee and Python
  • EX1.3 - Documentation affects understanding
• EX2 - Ecosystem
  • EX2.1 - Tool support affects understanding
  • EX2.2 - Community support
  • EX2.3 - Availability of third-party educational resources