To calculate the accuracy, recall, and specificity from the confusion matrix provided, we use the following formulas:

Confusion Matrix:

Actually Rotten: 45 (True Positive), 8 (False Positive)

Actually Fresh: 5 (False Negative), 42 (True Negative)

Accuracy:

Accuracy is the proportion of true results (both true positives and true negatives) in the total population.

Formula: Accuracy=TP+TNTP+TN+FP+FN\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}Accuracy=TP+TN+FP+FNTP+TN​

Calculation: Accuracy=45+4245+42+8+5=87100=0.87\text{Accuracy} = \frac{45 + 42}{45 + 42 + 8 + 5} = \frac{87}{100} = 0.87Accuracy=45+42+8+545+42​=10087​=0.87

Recall (Sensitivity):

Recall is the proportion of true positive results in the total actual positives.

Formula: Recall=TPTP+FN\text{Recall} = \frac{TP}{TP + FN}Recall=TP+FNTP​

Calculation: Recall=4545+5=4550=0.9\text{Recall} = \frac{45}{45 + 5} = \frac{45}{50} = 0.9Recall=45+545​=5045​=0.9

Specificity:

Specificity is the proportion of true negative results in the total actual negatives.

Formula: Specificity=TNTN+FP\text{Specificity} = \frac{TN}{TN + FP}Specificity=TN+FPTN​

Calculation: Specificity=4242+8=4250=0.84\text{Specificity} = \frac{42}{42 + 8} = \frac{42}{50} = 0.84Specificity=42+842​=5042​=0.84

Therefore, the correct combinations of accuracy, recall, and specificity are 0.87, 0.9, and 0.84 respectively.

References:

ISTQB CT-AI Syllabus, Section 5.1, Confusion Matrix, provides detailed formulas and explanations for calculating various metrics including accuracy, recall, and specificity.

"ML Functional Performance Metrics" (ISTQB CT-AI Syllabus, Section 5).

Detecting biases in the ML pipeline involves various tests to ensure fairness and accuracy throughout the ML process.

Testing the distribution shift in the training data for inappropriate bias (A) : This involves checking if there is any shift in the data distribution that could lead to bias in the model. It is an important test but not the most direct method for detecting biases.

Test the model during model evaluation for data bias (B) : This is a critical stage where the model is evaluated to detect any biases in the data it was trained on. It directly addresses potential data biases in the model.

Testing the data pipeline for any sources for algorithmic bias (C) : This test is crucial as it helps identify biases that may originate from the data processing and transformation stages within the pipeline. Detecting sources of algorithmic bias ensures that the model does not inherit biases from these processes.

Check the input test data for potential sample bias (D) : While this is an important step, it focuses more on the input data and less on the overall data pipeline.

Hence, the most likely useful test to help detect different kinds of biases in the ML pipeline is B. Test the model during model evaluation for data bias .

References :

ISTQB CT-AI Syllabus Section 8.3 on Testing for Algorithmic, Sample, and Inappropriate Bias discusses various tests that can be performed to detect biases at different stages of the ML pipeline.

Sample Exam Questions document, Question #32 highlights the importance of evaluating the model for biases.

The question asks which deficiency is most likely to be discovered by the test expert given the scenario of poor real-world performance despite good isolated accuracy.

A lack of similarity between the training and testing data (A) : This is a common issue in ML where the model performs well on training data but poorly on real-world data due to a lack of representativeness in the training data. This leads to poor generalization to new, unseen data.

The input data has not been tested for quality prior to use for testing (B) : While data quality is important, this option is less likely to be the primary reason for the described issue compared to the representativeness of training data.

A lack of focus on choosing the right functional-performance metrics (C) : Proper metrics are crucial, but the issue described seems more related to the data mismatch rather than metric selection.

A lack of focus on non-functional requirements testing (D) : Non-functional requirements are important, but the scenario specifically mentions issues with detecting real cancer cases, pointing more towards data issues.

References :

ISTQB CT-AI Syllabus Section 4.2 on Training, Validation, and Test Datasets emphasizes the importance of using representative datasets to ensure the model generalizes well to real-world data.

Sample Exam Questions document, Question #40 addresses issues related to data representativeness and model generalization.

Setting model and algorithm hyperparameters is an essential step in the machine learning workflow, primarily occurring during the tuning phase.

Evaluating the model (A) : This stage involves assessing the model's performance using metrics and does not typically include the setting of hyperparameters.

Deploying the model (B) : Deployment is the stage where the model is put into production and used in real-world applications. Hyperparameters should already be set before this stage.

Tuning the model (C) : This is the correct stage where hyperparameters are set. Tuning involves adjusting the hyperparameters to optimize the model's performance.

Data testing (D) : Data testing involves ensuring the quality and integrity of the data used for training and testing the model. It does not include setting hyperparameters.

Hence, the most appropriate stage of the ML workflow to set model and algorithm hyperparameters is C. Tuning the model .

References :

ISTQB CT-AI Syllabus Section 3.2 on the ML Workflow outlines the different stages of the ML process, including the tuning phase where hyperparameters are set.

Sample Exam Questions document, Question #31 specifically addresses the stage in the ML workflow where hyperparameters are configured.

A. The difficulty of defining criteria for improvement before the model can be accepted.

Defining criteria for improvement is a challenge in the acceptance of AI models, but it is not directly related to the performance drop in real-world scenarios. It relates more to the evaluation and deployment phase rather than affecting the model's real-time performance post-deployment.

B. The fast pace of change did not allow sufficient time for testing.

This can significantly affect the model's performance. If the system is self-learning, it needs to adapt quickly, and insufficient testing time can lead to incomplete learning and poor performance.

C. The unknown nature and insufficient specification of the operating environment might have caused the poor performance.

This is highly likely to affect performance. Self-learning AI systems require detailed specifications of the operating environment to adapt and learn effectively. If the environment is insufficiently specified, the model may fail to perform accurately in real-world scenarios.

D. There was an algorithmic bias in the AI system.

Algorithmic bias can significantly impact the performance of AI systems. If the model has biases, it will not perform well across different scenarios and data distributions.

Given the context of the self-learning nature and the need for real-time adaptability, option A is least likely to describe the fall in performance because it deals with acceptance criteria rather than real-time performance issues​​.

For AI-enabled medical devices, the most required aspects for certification are safety, transparency, and side effects. Here’s why:

Safety (Aspect III) : Critical for ensuring that the AI system does not cause harm to patients.

Transparency (Aspect IV) : Important for understanding and verifying the decisions made by the AI system.

Side Effects (Aspect V) : Necessary to identify and mitigate any unintended consequences of the AI system.

Why Not Other Options :

Autonomy and Maintainability (Aspects I and II) : While important, they are secondary to the immediate concerns of safety, transparency, and managing side effects in life-critical processes.

References: This explanation is aligned with the critical quality characteristics for AI-based systems as mentioned in the ISTQB CT-AI syllabus, focusing on the certification of medical devices​​.

 Type of Testing for Stock-Price Prediction Models: Concept drift refers to the change in the statistical properties of the target variable over time. Severe weather storms causing massive losses in coffee production and affecting stock prices would require testing for concept drift to ensure that the model adapts to new patterns in data over time.

 Reference: ISTQB_CT-AI_Syllabus_v1.0, Section 7.6 Testing for Concept Drift, which explains the need to test for concept drift in models that might be affected by changing external factors​​.