Machine Learning Academy · 5 / 5

Add a training-data quality gate

Consolidate the model and leave publication for a reviewed training result.

YOUR TASK

Make this happen

3 steps

Add a second normalization stage named nonnegative-quantity loading scripts/quality.lua. Drop nil or negative quantity, return input otherwise. Keep autoPublish false and all earlier stages for platform evaluation.
Keep the working capabilities from the previous lesson and connect the new pieces to the same solution.
Check the complete solution, then inspect any unmet requirements.

THE IDEA BEHIND IT

What this step adds

Add a second normalization stage named nonnegative-quantity loading scripts/quality.lua. Drop nil or negative quantity, return input otherwise. Keep autoPublish false and all earlier stages for platform evaluation.

Files to work in

main.cococo, model.cococo, quality.cococo, scripts/quality.lua. Other files are the working baseline. New files include named resource skeletons and TODO guidance. Complete their fields and connect them using the objectives, schema completion and reference guide.

Authoring and operation

The virtual document serializes to the platform authoring source. Training, immutable model versions and publication are separate operations; this exercise does none of them.

Complete the experiment

Preview columns and missing values, train on the intended cohort, inspect failures and metrics, then evaluate on separate data. Reject the model if it cannot improve on the baseline. Classification uses categorical targets; forecasting requires a time column and a separate time-series specification.

Checks validate native model configuration and lesson requirements. They do not run SQL, normalize rows, train a model, score predictions or establish predictive quality.

Files to change: main.cococo, model.cococo, quality.cococo, scripts/quality.lua

What you’re building: Author a reviewable job-quantity regression model with explicit features, missing-value handling and deliberate publication.

This step starts with the earlier solution and one new challenge. Each lesson keeps its own draft.

Read the starter files

main.cococo

ml_model_document "main" {
  format = "1"
  name = "job-quantity"
  backend = "xgboost"
  spec = {
    tabular = ml_model_tabular.quantity
  }
  dataQuerySQL = file("queries/training.sql")
}

model.cococo

ml_model_tabular "quantity" {
  task = "REGRESSION"
  targetColumn = "quantity"
  featureColumns = [ml_model_feature.priority]
  normalizationStages = [ml_model_normalization_stage.clean]
  hyperparameters = ml_model_hyperparameters.baseline
}

features.cococo

ml_model_feature "priority" {
  column = "priority"
}

queries/training.sql

SELECT quantity, priority FROM jobs WHERE quantity IS NOT NULL LIMIT 1000

normalize.cococo

ml_model_normalization_stage "clean" {
  name = "require-priority"
  source = file("scripts/normalize.lua")
}

scripts/normalize.lua

if input.priority == nil then return nil end
return { quantity = input.quantity, priority = input.priority }

tuning.cococo

ml_model_hyperparameters "baseline" {
  maxDepth = 3
  eta = 0.1
  rounds = 100
  evalMetric = "rmse"
}

quality.cococo

ml_model_normalization_stage "valid_quantity" {
  name = "nonnegative-quantity"
  # Source file supplied for this field: scripts/quality.lua
  # TODO: attach it with source = file(...).
}

scripts/quality.lua

-- input is the row from the previous normalization stage.
-- TODO: reject missing or negative quantity before returning accepted rows.
return input

Supplied supporting files

Reference: machine-learning · authoring