Quantitative Data and Data Collection Methods in Psychology

During the lecture, students learn the hierarchy of quantitative research data (from primary to tertiary data) and understand its application in psychological research. Measurement scales and their impact on subsequent statistical analysis are analyzed in detail. Particular attention is devoted to the diversity of data collection methods and modern digital opportunities, while consolidating the understanding of personal data protection (GDPR) and research ethics standards.

Lecture Sub-topics:

1. Data Taxonomy and Sources:

• Primary Data: Data collected firsthand for a specific research project.
• Secondary Data: Data collected by other researchers, national statistics portals, and Open Science databases (e.g., OSF, Mendeley Data).
• Tertiary Data: Results of meta-analyses and systematic reviews as data sources.

2. Levels of Measurement and Scale Types:

• S.S. Stevens’ NOIR Classification: Nominal, Ordinal, Interval, and Ratio scales.
• Variable Types: Qualitative (categorical) vs. Quantitative (metric); Discrete vs. Continuous variables.
• Specialized Scales in Psychology: Likert-type scales (their nature: ordinal vs. interval), Visual Analogue Scales (VAS), and Semantic Differential.

3. Data Collection Methods and Tools:

• Self-report Instruments: Surveys and questionnaires.
• Observation Protocols: Coding and quantification of behavior.
• Objective Measures: Reaction time, physiological measurements.
• Digital Phenotyping: Data retrieved from smart devices and sensors.

4. Data Protection and Research Ethics:

• General Data Protection Regulation (GDPR): Data minimization, transparency, and integrity.
• Informed Consent: Structure and mandatory elements.
• Data Security: Anonymization, pseudonymization, and data storage protocols.

Objective: To consolidate the ability to classify data and to locate existing data resources.

1. Article Analysis (Group Work): Each group receives one short, high-quality research article (or abstract).

Task: Complete a structured worksheet:

• What is the research object/subject?
• Identify the main variables.
• Determine the measurement level (scale) and type (quantitative/qualitative) for each variable.
• Indicate whether the data used are primary or secondary.

2. Secondary Data "Scavenger Hunt": Using their computers, students must navigate to an open-access database (e.g., European Social Survey or the Latvian Open Data Portal).

Task: Locate one dataset related to psychology or social sciences and describe the measurement scales predominantly used in that dataset.

During the lecture, students explore the nature of a psychological test as a standardized measurement instrument, distinguishing scientific approaches from pseudo-psychological methods. A comprehensive overview of test classification by purpose and format is provided, alongside an analysis of the core characteristics of a sound psychometric test: objectivity, standardization, and utility. Students are introduced to the historical development of psychometrics and master the fundamental axiom of Classical Test Theory (CTT): X = T + E. Particular attention is devoted to types of measurement error (systematic and random error), fostering an understanding of how environmental, respondent, and instrumental factors influence the precision of observed scores. These insights serve as a foundation for the practical session, where students analyze professional test batteries and perform a measurement error "audit."

Objective: To conduct an in-depth study of existing tests and, based on that analysis, develop a well-reasoned structure for an original test.

Group Work Procedure and Tasks:

1. Targeted Search and Selection:

• Groups select a construct to be measured (e.g., "Digital Device Use Anxiety").
• Locate at least two existing instruments. Analyze the rationale provided by the authors: Why was this test originally created? What specific problem did it address?

2. Structural and Psychometric Revision:

• Item Count and Content: How many items are in each scale? Are they concise or lengthy?
• Response Format: Why was this specific scale chosen (e.g., a 4-point forced-choice scale vs. a 7-point Likert scale)?
• Identification of Limitations: Students examine the "Limitations" section of the original research. Are the tests too long? Are they outdated? Were they validated only in a specific population?

During the lecture, students explore the scientific cycle of psychometric test development, from the initial theoretical concept to the first draft of test items. The three primary approaches to test construction—deductive, inductive, and integrative—are analyzed, emphasizing their differences in achieving research objectives. Students learn to perform construct operationalization, select the most appropriate response scale formats, and identify common types of response bias. The lecture concludes with an examination of content and face validity as the primary quality control mechanisms in the initial stages of test development.

Lecture Sub-topics:

1. Strategies for Test Construction:

• Deductive (Theory-driven) Approach: Building items based on existing theoretical frameworks.
• Inductive (Data-driven/Empirical) Approach: Using statistical relationships to derive dimensions.
• Integrative Approach: Combining theoretical foundations with empirical evidence.

2. From Construct to Measurement:

• Nominal and operational definitions of a construct.
• Identification of indicators and domains.

3. Item Formulation and Scales:

• Response Scale Formats: Likert, semantic differential, forced-choice, and dichotomous scales.
• Item Writing Hygiene: Linguistic precision and clarity.
• Reverse-scored Items: Purpose and implementation.

4. Response Bias:

• Social desirability, acquiescence bias, extreme response bias, and central tendency bias.
• Strategies for mitigating response bias during the design phase.

5. Initial Validation:

• Content Validity: Expert judgment and review.
• Face Validity: The respondent's perception of the instrument's relevance.

Objective: To practically develop the first set of test items and perform an initial quality assessment.

Session Procedure:

1. Group Work – Item Formulation:

• Based on the "test vision" from the previous session, groups formulate the operational definition of their construct.
• Select and justify the chosen response scale format.
• Create an initial list of items (e.g., 12–15 items).
• Using a consensus approach, the group agrees on which items to retain for further evaluation.

2. Content Validity Simulation:

• Groups "exchange" their items. Group A acts as "experts" for Group B’s test.
• Experts rate the relevance of each item to the operational definition (e.g., using a 4-point scale: "1 – Irrelevant", "2 – Slightly Relevant", "3 – Relevant", "4 – Highly Relevant").

3. Face Validity Check:

• If students in the classroom match the target audience, they provide feedback on the clarity, comprehensibility, and whether the items "really look like what they are supposed to measure."
• General feedback is provided on linguistic phrasing and respondent burden.

4. Reflection and Refinement:

• Groups receive their items back with comments and implement corrections.
• A finalized set of items is developed for future empirical evaluation.

Pass/Fail Submission:

Each small group submits a "Test Vision & Initial Item Set" report, which includes:

• A brief operational definition of the measured construct.
• A rationale for the necessity of the test development.
• The initial list of items with expert content validity ratings.
• Conclusions on which items are retained for further empirical approbation.

During the lecture, students acquire the methodological framework for adapting psychological tests developed in foreign languages to a different linguistic and cultural environment. Primary emphasis is placed on the International Test Commission (ITC) guidelines and the application of TARES standards to ensure measurement objectivity. Students are introduced to the various types of equivalence and their significance, as well as learning to identify specific biases that may arise as a result of inaccurate adaptation.

Lecture Sub-topics:

1. Introduction to Adaptation:

• The distinction between translation and adaptation.
• Rationale for adaptation (cultural, linguistic, and conceptual differences).

2. Stages of Adaptation According to ITC Guidelines:

• Preparation: Obtaining permission from the original test authors.
• The Translation Process: Forward translation and back-translation protocols.
• Synthesis and Expert Committee: Resolving discrepancies in translations.
• Approbation: Pilot studies and cognitive interviews with respondents.

3. Equivalence and Types of Bias:

• Equivalence: Linguistic, functional, conceptual, and metric equivalence.
• Bias: Construct bias, method bias (e.g., response styles), and item bias (e.g., culture-specific idioms).

4. Quality Frameworks:

• TARES Standards: Assessing the usability of the adapted test.
• EFPA (European Federation of Psychologists' Associations): A brief overview of the test evaluation model.

5. Data Collection Planning and Ethics:

• Sample size requirements for approbation.
• Re-evaluating GDPR: Ensuring data security throughout the adaptation process.

Objective: To perform the adaptation of a selected international instrument and to prepare for the empirical testing of both instruments (the original student-developed test and the adapted one).

Session Procedure:

1. Instrument Selection and Rationale:

• From the tests researched in the 2nd session, each group selects one that is most closely related to their chosen construct.
• This test will serve as the "gold standard" or a comparative tool for evaluating the validity of their original test.

2. Translation and Adaptation Workshop:

• Forward Translation: The group translates the test items into Latvian.
• Expert Review: An internal group discussion on conceptual equivalence. For example, is "anxious" better translated as "satraukts", "bažīgs", or "trauksmains" within the context of the specific construct?
• Cultural Adaptation: Are all examples from the original test understandable to a Latvian audience? If not, how can they be replaced without altering the item's underlying meaning?

3. Development of the Research Instrument (Questionnaire):

• Students prepare a unified survey (e.g., Google Forms or MS Forms) including:
  • Demographic questions (age, gender, etc.).
  • The original student-developed test (from Session 3).
  • The adapted international test (from Session 4).
• This design will enable correlation analysis for validity testing during the practical work in Sessions 5 and 6.

4. Data Collection Plan:

• Groups agree on a recruitment strategy (e.g., each student must survey 5–10 acquaintances/peers) for the approbation process.
• Ethical principles are reinforced: participation is voluntary and anonymous.

During the lecture, students learn methods for the empirical quality assessment of psychological test items. Item difficulty (response index) and discrimination indicators are analyzed in detail, teaching students how to interpret the distribution of responses and identify "non-performing" items. The lecture concludes with an introduction to test reliability using internal consistency measures and demonstrates the link between individual item quality and overall scale reliability.

Lecture Sub-topics:

1. Data Cleaning and Preparation:

• Handling missing values.
• Recoding reverse-coded items.

2. Item Response and Distribution Analysis:

• Item Difficulty (Response) Index (M / p): Determining what proportion of respondents selected the "keyed" or "measured" response.
• Response Distribution Analysis: Frequency tables and histograms. Recognizing "ceiling" or "floor" effects (when responses lack variance).

3. Item Discrimination Indicators:

• Item Discrimination Index (D): The ability to distinguish between respondents with high and low levels of the measured trait (comparing extreme groups).
• Discrimination Coefficient (r): Item-total correlation. Critical threshold values (typically r > 0.20).

4. Introduction to Scale Reliability:

• Internal Consistency and Cronbach’s Alpha (alpha): Measuring the interrelatedness of items.
• "Alpha if Item Deleted": Understanding how specific item parameters (difficulty and discrimination) influence the reliability of the entire test.

Objective: To perform an in-depth empirical analysis of the collected (or simulated) data and to make reasoned decisions regarding test shortening and refinement.

Session Procedure:

1. Working with Software (JASP/Jamovi):

• Students import their survey data.
• Perform data preparation (e.g., recoding reverse-scored items).

2. Item "Audit":

• Students calculate response indices (means/proportions) and discrimination coefficients for all items within each scale or subscale.
• Task: Identify items that exhibit:
  • Critically low discrimination (r < 0.20).
  • Highly asymmetrical distributions (e.g., floor or ceiling effects where most responses are clustered at one end of the scale).

3. Test Optimization:

• Students experiment with the "Alpha if item deleted" tool.
• They practice removing the weakest items to observe how the overall reliability (internal consistency) of the test improves.

4. Conclusions and Discussion:

• Groups discuss: "Why exactly did this specific question fail? Was it due to the translation, the phrasing, or the underlying construct itself?"

Submission (Pass/Fail): "Item Analysis Protocol"

Each group submits a protocol containing:

• A summary table with all calculated indicators (p/M, r, and alpha if item deleted).
• A 3–4 sentence summary explaining which items are retained for the final version and the rationale behind these decisions.

During the lecture, students deepen their understanding of psychological test reliability as the stability and precision of a measurement. Various methods for assessing reliability are analyzed (test-retest, parallel forms, split-half, and internal consistency), alongside their suitability for different types of tests.

Lecture Sub-topics:

1. The Concept of Reliability and CTT:

• Re-evaluating X = T + E. Reliability as the ratio of true score variance to total observed variance.
• Factors influencing reliability (test length, sample heterogeneity).

2. Methods for Assessing Reliability:

• Stability (Test-retest Reliability): The time factor and the memory/practice effect.
• Equivalence (Parallel Forms Reliability): When is it necessary to develop alternate versions?
• Internal Consistency: Cronbach’s alpha (alpha) and McDonald’s omega (omega) – understanding why omega is becoming the preferred indicator in modern psychometrics.
• Inter-rater Reliability: Essential for observational and projective methods (Cohen’s kappa).

3. Interpretation of Reliability Coefficients:

• Threshold values in research (0.70) versus individual clinical diagnostics (0.90).
• TARES standards for reporting reliability.

Objective: To practically calculate and interpret various reliability indicators for the student's mini-project.

Session Procedure:

1. Reliability Calculations in Software:

• Students calculate internal consistency (Alpha and Omega) for both their original and adapted tests.
• If data are available (or simulated), test-retest correlation is calculated.

2. SEM (Standard Error of Measurement) Workshop:

• Using the obtained reliability coefficient and the standard deviation, students calculate the SEM for their test.
• Task: Create a sample for an individual report: "If Client X scores 45 points, within what interval does their true score lie?"

3. Comparative Analysis:

• Groups compare: which version of the test (original or adapted) shows higher reliability? Why?

4. Report Preparation:

• Record the reliability indicators for the final project, formatting them in a table and providing a structured description.

During the lecture, students explore the modern concept of validity as a unified body of evidence supporting the appropriateness of test score interpretations. Evidence of construct validity (convergent and discriminant validity) and types of criterion validity (predictive, concurrent, incremental, and differential validity) are analyzed in detail. Students learn to evaluate evidence confirming that a test indeed measures the intended psychological construct and is capable of predicting real-world behavior or other clinical indicators.

Lecture Sub-topics:

1. Evolution of the Validity Concept:

• From the "Three Pillars" (content, criterion, construct) to a unified validity framework.
• The relationship between reliability and validity (reliability as a necessary but insufficient condition).

2. Construct Validity:

• Convergent Validity: Correspondence with other instruments measuring the same construct.
• Discriminant (Divergent) Validity: Low correlation with instruments measuring unrelated constructs.

3. Types of Criterion Validity:

• Concurrent Validity: Relationship with a criterion measured at the same point in time.
• Predictive Validity: The ability to forecast future behavior or outcomes.
• Incremental Validity: Assessing whether the new test provides additional information beyond existing instruments.
• Differential Validity: The ability to distinguish between different groups (e.g., a clinical group vs. a normative group).

4. Interpretation of Validity Coefficients:

• Utilizing correlation matrices.
• Limitations (e.g., the impact of criterion unreliability on validity coefficients).

Objective: To master the methodology for calculating various types of validity and to perform an initial validity assessment for the student's mini-project.

Session Procedure:

Part 1: Working with the Instructor's Dataset

• Calculation of Criterion Validity: Using data from a larger study, students conduct a correlation analysis between a new scale and several external criteria (e.g., academic achievement, clinical diagnosis, or behavioral indicators).
• Testing Differential Validity: Using t-tests or ANOVA, students learn to determine whether the test significantly distinguishes between different groups (e.g., a control group vs. a clinical group).
• Demonstration of Incremental Validity: A brief conceptual overview of hierarchical regression to see if the new scale improves predictive accuracy beyond existing variables.

Part 2: Working with Self-Collected Data

• Testing Concurrent Validity: Students calculate the correlation in their own data between the original scale they developed and the adapted international test.
• Data Visualization: Creation and interpretation of correlation scatterplots—is the relationship linear? Are there visible outliers?
• Initial Construct Validity Analysis: Students compare their test with another variable included in the survey (e.g., age or gender) to test for discriminant validity (low correlation where theoretically no relationship is expected).

Students record the indicators that support the validity of their test, ensuring they are correctly formatted and described for the final project report.

Lecture Annotation: In this final lecture of the psychometrics course, students learn methods for determining a test’s internal structure and diagnostic value. An overview of factor analysis (EFA and CFA) is provided, explaining how latent variables structure observed data. Students are introduced to clinical psychometric indicators (sensitivity, specificity, and the ROC curve) and modern measurement theories, such as Item Response Theory (IRT) and the psychological network approach.

Lecture Sub-topics:

1. Factor Analysis: Structural Validity:

• EFA (Exploratory Factor Analysis): How do items cluster? Interpretation of factor loadings.
• CFA (Confirmatory Factor Analysis): Testing the theoretical model. Key fit indices ($CFI$, $RMSEA$).

2. Clinical Utility and Diagnostic Accuracy:

• Sensitivity vs. Specificity: Balancing true positive and true negative rates.
• ROC Curve: Determining the optimal cut-off point for diagnostic decisions.

3. Development of Norms:

• Standardized scores ($z$, $T$, stens, percentiles) and their application.

4. Expanded Horizons: IRT and Network Psychometrics:

• IRT (Item Response Theory): Item difficulty and discrimination as non-linear functions of latent ability.
• Network Approach: Viewing psychological constructs as a system of interacting symptoms rather than a single common cause.

5. Preparing a Psychometric Report:

• Report Structure: From construct definition to norms.
• Data Visualization and Table Formatting: Following APA (American Psychological Association) style.

Objective: To master the practical steps of conducting factor analysis and to prepare for the development of the final project report.

Session Procedure:

1. Working with Instructor's Data – EFA:

• Students perform Exploratory Factor Analysis (EFA) on a large dataset using JASP or Jamovi.
• Task: Determine the number of factors (using a Scree plot or Parallel Analysis) and interpret factor loadings. Identify "cross-loading" items.

2. Working with Instructor's Data – CFA:

• Demonstration: Students attempt to confirm the previously identified factor structure.
• Task: Interpret the primary model fit indices—does the model "fit" the data? (Evaluating CFI, TLI, and RMSEA).

3. Preparation for the Final Submission:

• Students work in their groups to compile all results from previous sessions (Sessions 2–7) into a single, unified structure.
• Consultations are provided regarding the formal requirements and standards for the final report.

4. Conclusion and Feedback:

• A comprehensive summary of the course and information regarding the evaluation criteria for the final presentation and submission.

During the lecture, students learn the hierarchy of quantitative research data (from primary to tertiary data) and understand its application in psychological research. Measurement scales and their impact on subsequent statistical analysis are analyzed in detail. Particular attention is devoted to the diversity of data collection methods and modern digital opportunities, while consolidating the understanding of personal data protection (GDPR) and research ethics standards.

Lecture Sub-topics:

1. Data Taxonomy and Sources:

• Primary Data: Data collected firsthand for a specific research project.
• Secondary Data: Data collected by other researchers, national statistics portals, and Open Science databases (e.g., OSF, Mendeley Data).
• Tertiary Data: Results of meta-analyses and systematic reviews as data sources.

2. Levels of Measurement and Scale Types:

• S.S. Stevens’ NOIR Classification: Nominal, Ordinal, Interval, and Ratio scales.
• Variable Types: Qualitative (categorical) vs. Quantitative (metric); Discrete vs. Continuous variables.
• Specialized Scales in Psychology: Likert-type scales (their nature: ordinal vs. interval), Visual Analogue Scales (VAS), and Semantic Differential.

3. Data Collection Methods and Tools:

• Self-report Instruments: Surveys and questionnaires.
• Observation Protocols: Coding and quantification of behavior.
• Objective Measures: Reaction time, physiological measurements.
• Digital Phenotyping: Data retrieved from smart devices and sensors.

4. Data Protection and Research Ethics:

• General Data Protection Regulation (GDPR): Data minimization, transparency, and integrity.
• Informed Consent: Structure and mandatory elements.
• Data Security: Anonymization, pseudonymization, and data storage protocols.

Objective: To consolidate the ability to classify data and to locate existing data resources.

1. Article Analysis (Group Work): Each group receives one short, high-quality research article (or abstract).

Task: Complete a structured worksheet:

• What is the research object/subject?
• Identify the main variables.
• Determine the measurement level (scale) and type (quantitative/qualitative) for each variable.
• Indicate whether the data used are primary or secondary.

2. Secondary Data "Scavenger Hunt": Using their computers, students must navigate to an open-access database (e.g., European Social Survey or the Latvian Open Data Portal).

Task: Locate one dataset related to psychology or social sciences and describe the measurement scales predominantly used in that dataset.

During the lecture, students explore the nature of a psychological test as a standardized measurement instrument, distinguishing scientific approaches from pseudo-psychological methods. A comprehensive overview of test classification by purpose and format is provided, alongside an analysis of the core characteristics of a sound psychometric test: objectivity, standardization, and utility. Students are introduced to the historical development of psychometrics and master the fundamental axiom of Classical Test Theory (CTT): X = T + E. Particular attention is devoted to types of measurement error (systematic and random error), fostering an understanding of how environmental, respondent, and instrumental factors influence the precision of observed scores. These insights serve as a foundation for the practical session, where students analyze professional test batteries and perform a measurement error "audit."

Objective: To conduct an in-depth study of existing tests and, based on that analysis, develop a well-reasoned structure for an original test.

Group Work Procedure and Tasks:

1. Targeted Search and Selection:

• Groups select a construct to be measured (e.g., "Digital Device Use Anxiety").
• Locate at least two existing instruments. Analyze the rationale provided by the authors: Why was this test originally created? What specific problem did it address?

2. Structural and Psychometric Revision:

• Item Count and Content: How many items are in each scale? Are they concise or lengthy?
• Response Format: Why was this specific scale chosen (e.g., a 4-point forced-choice scale vs. a 7-point Likert scale)?
• Identification of Limitations: Students examine the "Limitations" section of the original research. Are the tests too long? Are they outdated? Were they validated only in a specific population?

During the lecture, students explore the scientific cycle of psychometric test development, from the initial theoretical concept to the first draft of test items. The three primary approaches to test construction—deductive, inductive, and integrative—are analyzed, emphasizing their differences in achieving research objectives. Students learn to perform construct operationalization, select the most appropriate response scale formats, and identify common types of response bias. The lecture concludes with an examination of content and face validity as the primary quality control mechanisms in the initial stages of test development.

Lecture Sub-topics:

1. Strategies for Test Construction:

• Deductive (Theory-driven) Approach: Building items based on existing theoretical frameworks.
• Inductive (Data-driven/Empirical) Approach: Using statistical relationships to derive dimensions.
• Integrative Approach: Combining theoretical foundations with empirical evidence.

2. From Construct to Measurement:

• Nominal and operational definitions of a construct.
• Identification of indicators and domains.

3. Item Formulation and Scales:

• Response Scale Formats: Likert, semantic differential, forced-choice, and dichotomous scales.
• Item Writing Hygiene: Linguistic precision and clarity.
• Reverse-scored Items: Purpose and implementation.

4. Response Bias:

• Social desirability, acquiescence bias, extreme response bias, and central tendency bias.
• Strategies for mitigating response bias during the design phase.

5. Initial Validation:

• Content Validity: Expert judgment and review.
• Face Validity: The respondent's perception of the instrument's relevance.

Objective: To practically develop the first set of test items and perform an initial quality assessment.

Session Procedure:

1. Group Work – Item Formulation:

• Based on the "test vision" from the previous session, groups formulate the operational definition of their construct.
• Select and justify the chosen response scale format.
• Create an initial list of items (e.g., 12–15 items).
• Using a consensus approach, the group agrees on which items to retain for further evaluation.

2. Content Validity Simulation:

• Groups "exchange" their items. Group A acts as "experts" for Group B’s test.
• Experts rate the relevance of each item to the operational definition (e.g., using a 4-point scale: "1 – Irrelevant", "2 – Slightly Relevant", "3 – Relevant", "4 – Highly Relevant").

3. Face Validity Check:

• If students in the classroom match the target audience, they provide feedback on the clarity, comprehensibility, and whether the items "really look like what they are supposed to measure."
• General feedback is provided on linguistic phrasing and respondent burden.

4. Reflection and Refinement:

• Groups receive their items back with comments and implement corrections.
• A finalized set of items is developed for future empirical evaluation.

Pass/Fail Submission:

Each small group submits a "Test Vision & Initial Item Set" report, which includes:

• A brief operational definition of the measured construct.
• A rationale for the necessity of the test development.
• The initial list of items with expert content validity ratings.
• Conclusions on which items are retained for further empirical approbation.

During the lecture, students acquire the methodological framework for adapting psychological tests developed in foreign languages to a different linguistic and cultural environment. Primary emphasis is placed on the International Test Commission (ITC) guidelines and the application of TARES standards to ensure measurement objectivity. Students are introduced to the various types of equivalence and their significance, as well as learning to identify specific biases that may arise as a result of inaccurate adaptation.

Lecture Sub-topics:

1. Introduction to Adaptation:

• The distinction between translation and adaptation.
• Rationale for adaptation (cultural, linguistic, and conceptual differences).

2. Stages of Adaptation According to ITC Guidelines:

• Preparation: Obtaining permission from the original test authors.
• The Translation Process: Forward translation and back-translation protocols.
• Synthesis and Expert Committee: Resolving discrepancies in translations.
• Approbation: Pilot studies and cognitive interviews with respondents.

3. Equivalence and Types of Bias:

• Equivalence: Linguistic, functional, conceptual, and metric equivalence.
• Bias: Construct bias, method bias (e.g., response styles), and item bias (e.g., culture-specific idioms).

4. Quality Frameworks:

• TARES Standards: Assessing the usability of the adapted test.
• EFPA (European Federation of Psychologists' Associations): A brief overview of the test evaluation model.

5. Data Collection Planning and Ethics:

• Sample size requirements for approbation.
• Re-evaluating GDPR: Ensuring data security throughout the adaptation process.

Objective: To perform the adaptation of a selected international instrument and to prepare for the empirical testing of both instruments (the original student-developed test and the adapted one).

Session Procedure:

1. Instrument Selection and Rationale:

• From the tests researched in the 2nd session, each group selects one that is most closely related to their chosen construct.
• This test will serve as the "gold standard" or a comparative tool for evaluating the validity of their original test.

2. Translation and Adaptation Workshop:

• Forward Translation: The group translates the test items into Latvian.
• Expert Review: An internal group discussion on conceptual equivalence. For example, is "anxious" better translated as "satraukts", "bažīgs", or "trauksmains" within the context of the specific construct?
• Cultural Adaptation: Are all examples from the original test understandable to a Latvian audience? If not, how can they be replaced without altering the item's underlying meaning?

3. Development of the Research Instrument (Questionnaire):

• Students prepare a unified survey (e.g., Google Forms or MS Forms) including:
  • Demographic questions (age, gender, etc.).
  • The original student-developed test (from Session 3).
  • The adapted international test (from Session 4).
• This design will enable correlation analysis for validity testing during the practical work in Sessions 5 and 6.

4. Data Collection Plan:

• Groups agree on a recruitment strategy (e.g., each student must survey 5–10 acquaintances/peers) for the approbation process.
• Ethical principles are reinforced: participation is voluntary and anonymous.

During the lecture, students learn methods for the empirical quality assessment of psychological test items. Item difficulty (response index) and discrimination indicators are analyzed in detail, teaching students how to interpret the distribution of responses and identify "non-performing" items. The lecture concludes with an introduction to test reliability using internal consistency measures and demonstrates the link between individual item quality and overall scale reliability.

Lecture Sub-topics:

1. Data Cleaning and Preparation:

• Handling missing values.
• Recoding reverse-coded items.

2. Item Response and Distribution Analysis:

• Item Difficulty (Response) Index (M / p): Determining what proportion of respondents selected the "keyed" or "measured" response.
• Response Distribution Analysis: Frequency tables and histograms. Recognizing "ceiling" or "floor" effects (when responses lack variance).

3. Item Discrimination Indicators:

• Item Discrimination Index (D): The ability to distinguish between respondents with high and low levels of the measured trait (comparing extreme groups).
• Discrimination Coefficient (r): Item-total correlation. Critical threshold values (typically r > 0.20).

4. Introduction to Scale Reliability:

• Internal Consistency and Cronbach’s Alpha (alpha): Measuring the interrelatedness of items.
• "Alpha if Item Deleted": Understanding how specific item parameters (difficulty and discrimination) influence the reliability of the entire test.

Objective: To perform an in-depth empirical analysis of the collected (or simulated) data and to make reasoned decisions regarding test shortening and refinement.

Session Procedure:

1. Working with Software (JASP/Jamovi):

• Students import their survey data.
• Perform data preparation (e.g., recoding reverse-scored items).

2. Item "Audit":

• Students calculate response indices (means/proportions) and discrimination coefficients for all items within each scale or subscale.
• Task: Identify items that exhibit:
  • Critically low discrimination (r < 0.20).
  • Highly asymmetrical distributions (e.g., floor or ceiling effects where most responses are clustered at one end of the scale).

3. Test Optimization:

• Students experiment with the "Alpha if item deleted" tool.
• They practice removing the weakest items to observe how the overall reliability (internal consistency) of the test improves.

4. Conclusions and Discussion:

• Groups discuss: "Why exactly did this specific question fail? Was it due to the translation, the phrasing, or the underlying construct itself?"

Submission (Pass/Fail): "Item Analysis Protocol"

Each group submits a protocol containing:

• A summary table with all calculated indicators (p/M, r, and alpha if item deleted).
• A 3–4 sentence summary explaining which items are retained for the final version and the rationale behind these decisions.

During the lecture, students deepen their understanding of psychological test reliability as the stability and precision of a measurement. Various methods for assessing reliability are analyzed (test-retest, parallel forms, split-half, and internal consistency), alongside their suitability for different types of tests.

Lecture Sub-topics:

1. The Concept of Reliability and CTT:

• Re-evaluating X = T + E. Reliability as the ratio of true score variance to total observed variance.
• Factors influencing reliability (test length, sample heterogeneity).

2. Methods for Assessing Reliability:

• Stability (Test-retest Reliability): The time factor and the memory/practice effect.
• Equivalence (Parallel Forms Reliability): When is it necessary to develop alternate versions?
• Internal Consistency: Cronbach’s alpha (alpha) and McDonald’s omega (omega) – understanding why omega is becoming the preferred indicator in modern psychometrics.
• Inter-rater Reliability: Essential for observational and projective methods (Cohen’s kappa).

3. Interpretation of Reliability Coefficients:

• Threshold values in research (0.70) versus individual clinical diagnostics (0.90).
• TARES standards for reporting reliability.

Objective: To practically calculate and interpret various reliability indicators for the student's mini-project.

Session Procedure:

1. Reliability Calculations in Software:

• Students calculate internal consistency (Alpha and Omega) for both their original and adapted tests.
• If data are available (or simulated), test-retest correlation is calculated.

2. SEM (Standard Error of Measurement) Workshop:

• Using the obtained reliability coefficient and the standard deviation, students calculate the SEM for their test.
• Task: Create a sample for an individual report: "If Client X scores 45 points, within what interval does their true score lie?"

3. Comparative Analysis:

• Groups compare: which version of the test (original or adapted) shows higher reliability? Why?

4. Report Preparation:

• Record the reliability indicators for the final project, formatting them in a table and providing a structured description.

During the lecture, students explore the modern concept of validity as a unified body of evidence supporting the appropriateness of test score interpretations. Evidence of construct validity (convergent and discriminant validity) and types of criterion validity (predictive, concurrent, incremental, and differential validity) are analyzed in detail. Students learn to evaluate evidence confirming that a test indeed measures the intended psychological construct and is capable of predicting real-world behavior or other clinical indicators.

Lecture Sub-topics:

1. Evolution of the Validity Concept:

• From the "Three Pillars" (content, criterion, construct) to a unified validity framework.
• The relationship between reliability and validity (reliability as a necessary but insufficient condition).

2. Construct Validity:

• Convergent Validity: Correspondence with other instruments measuring the same construct.
• Discriminant (Divergent) Validity: Low correlation with instruments measuring unrelated constructs.

3. Types of Criterion Validity:

• Concurrent Validity: Relationship with a criterion measured at the same point in time.
• Predictive Validity: The ability to forecast future behavior or outcomes.
• Incremental Validity: Assessing whether the new test provides additional information beyond existing instruments.
• Differential Validity: The ability to distinguish between different groups (e.g., a clinical group vs. a normative group).

4. Interpretation of Validity Coefficients:

• Utilizing correlation matrices.
• Limitations (e.g., the impact of criterion unreliability on validity coefficients).

Objective: To master the methodology for calculating various types of validity and to perform an initial validity assessment for the student's mini-project.

Session Procedure:

Part 1: Working with the Instructor's Dataset

• Calculation of Criterion Validity: Using data from a larger study, students conduct a correlation analysis between a new scale and several external criteria (e.g., academic achievement, clinical diagnosis, or behavioral indicators).
• Testing Differential Validity: Using t-tests or ANOVA, students learn to determine whether the test significantly distinguishes between different groups (e.g., a control group vs. a clinical group).
• Demonstration of Incremental Validity: A brief conceptual overview of hierarchical regression to see if the new scale improves predictive accuracy beyond existing variables.

Part 2: Working with Self-Collected Data

• Testing Concurrent Validity: Students calculate the correlation in their own data between the original scale they developed and the adapted international test.
• Data Visualization: Creation and interpretation of correlation scatterplots—is the relationship linear? Are there visible outliers?
• Initial Construct Validity Analysis: Students compare their test with another variable included in the survey (e.g., age or gender) to test for discriminant validity (low correlation where theoretically no relationship is expected).

Students record the indicators that support the validity of their test, ensuring they are correctly formatted and described for the final project report.

Lecture Annotation: In this final lecture of the psychometrics course, students learn methods for determining a test’s internal structure and diagnostic value. An overview of factor analysis (EFA and CFA) is provided, explaining how latent variables structure observed data. Students are introduced to clinical psychometric indicators (sensitivity, specificity, and the ROC curve) and modern measurement theories, such as Item Response Theory (IRT) and the psychological network approach.

Lecture Sub-topics:

1. Factor Analysis: Structural Validity:

• EFA (Exploratory Factor Analysis): How do items cluster? Interpretation of factor loadings.
• CFA (Confirmatory Factor Analysis): Testing the theoretical model. Key fit indices ($CFI$, $RMSEA$).

2. Clinical Utility and Diagnostic Accuracy:

• Sensitivity vs. Specificity: Balancing true positive and true negative rates.
• ROC Curve: Determining the optimal cut-off point for diagnostic decisions.

3. Development of Norms:

• Standardized scores ($z$, $T$, stens, percentiles) and their application.

4. Expanded Horizons: IRT and Network Psychometrics:

• IRT (Item Response Theory): Item difficulty and discrimination as non-linear functions of latent ability.
• Network Approach: Viewing psychological constructs as a system of interacting symptoms rather than a single common cause.

5. Preparing a Psychometric Report:

• Report Structure: From construct definition to norms.
• Data Visualization and Table Formatting: Following APA (American Psychological Association) style.

Objective: To master the practical steps of conducting factor analysis and to prepare for the development of the final project report.

Session Procedure:

1. Working with Instructor's Data – EFA:

• Students perform Exploratory Factor Analysis (EFA) on a large dataset using JASP or Jamovi.
• Task: Determine the number of factors (using a Scree plot or Parallel Analysis) and interpret factor loadings. Identify "cross-loading" items.

2. Working with Instructor's Data – CFA:

• Demonstration: Students attempt to confirm the previously identified factor structure.
• Task: Interpret the primary model fit indices—does the model "fit" the data? (Evaluating CFI, TLI, and RMSEA).

3. Preparation for the Final Submission:

• Students work in their groups to compile all results from previous sessions (Sessions 2–7) into a single, unified structure.
• Consultations are provided regarding the formal requirements and standards for the final report.

4. Conclusion and Feedback:

• A comprehensive summary of the course and information regarding the evaluation criteria for the final presentation and submission.