cw.

Data engineer / Scotland

Callum WattMaking data
work.
Making systems
make sense.

From production data pipelines to adversarial AI research. I build, investigate, and explore the systems behind the signal.

Explore my work Enter the lab
Open to data engineering opportunities
A connected system: ingest, transform, validate01 / INGEST02 / TRANSFORM03 / VALIDATE

STRUCTURE IN THE COMPLEXITY

Previous experienceFanDuel · Data EngineerJune–September 2026
PostgraduateMSc FinTech · DistinctionAbertay University
FoundationBSc (Hons) Ethical HackingAbertay University

01 / Selected work

Built on curiosity.
Grounded in practice.

Engineering + research
Explore the decisions behind the work.

01 / CONCEPT DIAGRAM
Industry experience

Finding the signal in pipeline failures.

Turning validation failures into an operational view engineers could investigate.

SQLDatabricksPySpark
Explore Data quality & observability

FanDuel · Data Engineer · June–September 2026

What I worked on

Built a Databricks monitoring dashboard to explore event-validation failures by reason, source application and product, with trends over time and a separate development view. Used SQL to investigate behavioural engagement signals, check attribute consistency and normalise comparisons by on-screen position.

Engineering perspective

Made failures easier to inspect in context rather than treating every rejected event as the same problem. Supporting work included precomputed dashboard metrics and decommissioning legacy ingestion tasks with dependency checks and documentation.

Scope & limitations

The lab is a generic educational recreation using synthetic records. It does not reproduce FanDuel architecture, internal dashboards, production data or proprietary queries.

Source material reviewed

Supported by the CV and archived MSc dissertation. Only general engineering principles are presented publicly.

02 / ILLUSTRATIVE TRACE
Academic research

When intelligent agents meet vulnerable markets.

Explore original archived output from a reinforcement learning market experiment.

PythonGymnasiumPPO
Explore AI market simulation

Undergraduate honours project · MarketSimEnv / Stable-Baselines3

What I worked on

Implemented a custom Gymnasium environment with hold, buy and sell actions. The four-value observation contains simulated price, holdings, market trend and cash balance. The supplied simulation.py loads a saved PPO model or trains an MlpPolicy agent for five million timesteps, then records 100 runs of 100 ticks.

Engineering perspective

The environment explicitly models buy-driven price increases and post-sale drops. Recording Run, Tick, Price, Action, Balance and Shares makes the agent’s interactions inspectable. The archived CSV contains 10,000 observations across 100 complete runs.

Scope & limitations

A deliberately simplified, reward-shaped simulator, not evidence of profitable trading or real-world manipulation capability. Saved traces below are original synthetic research output. The separate lab uses a new mathematical model, not the trained PPO agent. The supplied code does not establish that surveillance classifiers were implemented.

Source material reviewed

Reviewed original simulation.py, its commented version, a saved model archive and simulation_output 0.9-0.98.csv. The viewer reads saved price data; it does not execute the model.

03 / CONCEPT DIAGRAM
MSc dissertation

Measure what people saw. Then what they did.

A data engineering framework for near-real-time behavioural analytics and meaningful funnel denominators.

LakehouseData modellingEvent quality
Explore Impression-based funnel analytics

MSc dissertation · Abertay University · 2026

What I worked on

Researched a layered data engineering framework that moves behavioural events through schema checks, refinement and analytical serving. The central modelling question is how to connect genuine on-screen impressions to subsequent engagement and outcomes, rather than counting every visit as equivalent exposure.

Engineering perspective

Examined near-real-time micro-batches versus continuous streaming, position-normalised engagement, deduplication, schema evolution and pre-aggregation for self-service analytics. Distinguishing content that was never seen from content that was seen but ignored changes how a funnel should be interpreted.

Scope & limitations

The dissertation concerns a placement context with confidentiality restrictions. This public summary uses general concepts only: no internal queries, architecture screenshots, company identifiers, private data or quantitative production results. Consent-dependent capture also means observed behaviour does not represent every user.

Source material reviewed

Reviewed the archived final ANF509 dissertation. This is distinct from the CMP511 gambling-risk machine learning coursework.

04 / CONCEPT DIAGRAM
Machine learning coursework

A high score is only useful if the target makes sense.

Separating descriptive risk labels from future escalation—and examining the precision–recall trade-off.

scikit-learnTemporal evaluationSHAP
Explore Behavioural risk modelling

CMP511 · Machine Learning & Artificial Intelligence · MSc coursework

What I worked on

Aggregated Bustabit transactions into player-level features: average and maximum bet, bet variability, loss rate, maximum-to-average bet ratio, activity count and log-linear stake trend. Compared Logistic Regression, Random Forest, SVC and KNN. A second task uses the first 70% of each player’s sequence as history and the remaining 30% to construct a future-escalation label.

Engineering perspective

The risk categories are engineered from percentile-based scores. A high classification accuracy can mainly reflect reconstruction of the labelling rule. In the future-escalation task, the report gives SVC recall of 0.857 with precision of 0.221, versus Random Forest recall of 0.200 and precision of 0.259. This exposes the cost of choosing sensitivity over false positives.

Scope & limitations

Labels are engineered, not clinically validated. The per-player 70/30 sequence split is followed by a stratified player-level train/test split; this is not a global forward-in-time deployment evaluation. These are reported coursework results, not independently rerun experiments or evidence of diagnosing gambling harm.

Source material reviewed

Reviewed bustabitML.py and CMP511_Final_Submission, including model comparison output on appendix page 31. The report also discusses K-Means, PCA, SHAP and error analysis.

05 / ILLUSTRATIVE TRACE
Academic project

From market data to a reproducible analysis.

A local-CSV pipeline with time-ordered evaluation and explicit feature engineering.

PythonRandom ForestTime series
Explore Financial analysis pipeline

MAT510 · Introduction to Data Science

What I worked on

The supplied newsim.py filters a local S&P 500 stock CSV by symbol and creates returns, 20- and 50-period moving averages, 20-period rolling standard deviation and a next-row closing-price target. It trains a Random Forest regressor and exports raw data, processed features and comparison charts.

Engineering perspective

The code uses an 80/20 split with shuffle=False, retaining input order. Its final configuration specifies 2,000 trees, maximum depth 20 and a fixed random seed. Evaluation uses MAE and RMSE; the report compares one-tree and 2,000-tree behaviour to discuss stability versus computation.

Scope & limitations

Input order is preserved rather than timestamps being explicitly sorted, so chronological ordering must be checked before use. The report notes lag during abrupt price changes and limited generalisation across stocks. An academic workflow, not a validated live forecasting service; no unsupported error scores are claimed.

Source material reviewed

Reviewed newsim.py and FinalMAT510Copy.pdf. The source dataset is not redistributed through the website.

06 / CONCEPT DIAGRAM
Design study / cybersecurity

Choosing a classifier starts with the failure modes.

A documented comparison of network-classification methods, pipeline stages and evaluation choices.

Random ForestSVMEvaluation design
Explore Network attack classifier design

CMP417 · Engineering Resilient Systems

What I worked on

The report compares Random Forest and SVM for network-traffic categories including DoS, exploits, reconnaissance and backdoors. It sets out a proposed pipeline from collection and preprocessing through feature engineering, training, evaluation and monitoring.

Engineering perspective

Examined noise, dimensionality, interpretability and computational cost. Proposed F1 alongside a confusion matrix to inspect which attack categories are missed or confused, rather than relying on overall accuracy alone.

Scope & limitations

The reviewed artefact is a classifier design and methodology report. It does not establish a trained implementation or measured detection performance, so this is presented as a design study, not a deployed intrusion-detection system.

Source material reviewed

Reviewed Machine Learning Resillient Systems.pdf. No trained classifier or evaluation output for this task was identified in the reviewed archive.

07 / CONCEPT DIAGRAM
Academic implementation

From a physical button to a stored event.

A blackjack application connecting GPIO interaction, game state and cloud-persisted results.

C++Raspberry PiAWS / MariaDB
Explore Raspberry Pi & cloud database

CMP408 · IoT & Cloud Secure Development

What I worked on

Developed C++ blackjack logic for a Raspberry Pi Zero W with draw, stand and double actions mapped to GPIO buttons through pigpio. The report describes AWS EC2 and RDS/MariaDB setup, database result logging and testing of hardware inputs and stored game outcomes.

Engineering perspective

Established terminal-based game logic before adding hardware input and database persistence. The report documents connectivity problems and an SSH fallback, showing the practical integration work between local software, a device and a cloud database.

Scope & limitations

An academic prototype, not a claim of production cloud scale or audited security. The report and source are not offered as downloads because they contain connection details; security configuration claims are not treated as independently verified.

Source material reviewed

Reviewed blackjackv3.cpp, IoT Report and the project poster. A demonstration video is present in the archive but is not published in this version.

02 / Production experience

Good data needs
good engineering.

At FanDuel, I worked with the pipelines, models and monitoring that turn behavioural events into usable analytics.

Data EngineerFanDuel · EdinburghJune–September 2026
01

Build for downstream use.

Worked on Databricks ETL/ELT pipelines with PySpark, Python and Airflow. Designed a model and supporting pipeline to precompute dashboard metrics for wider analytical use.

02

Make failures explainable.

Built an operational dashboard for schema-validation failures. Investigated patterns using SQL, with views by source, product, failure reason and time.

03

Maintain the whole system.

Decommissioned legacy ingestion tasks, checked data flows and dependencies, updated documentation and coordinated with downstream users.

Descriptions reflect my contribution. All public demos are independent, generic examples.

03 / The engineering lab

Don't just read.
Pull a thread.

Three small experiments.
Real interactions. Synthetic data.

Validate before you trust

One bad event.
A contained problem.

Choose a fault and run the validator. Invalid records are quarantined while clean events continue downstream.

SYNTHETIC EVENT PIPELINECLEAN
IngestValidateAnalytics
12received
12accepted
0quarantined

04 / A connected perspective

Different disciplines.
One systems mindset.

I'm Callum Watt, a data engineer from Inverness in the Scottish Highlands, now based in Dundee. My path connects ethical hacking, financial technology and production data infrastructure.

I’m interested in where systems break, what the data reveals, and how to make the result more reliable. That runs through my security projects, market simulations and work investigating data quality.

Find me on GitHub ↗
  1. 2021–2025 / Abertay University

    BSc (Hons) Ethical Hacking

    Security, adversarial thinking and AI market surveillance research.

  2. 2025–2026 / Abertay University

    MSc FinTech · Distinction

    Impression-based funnel analytics, financial systems and machine learning.

  3. June–September 2026 / FanDuel

    Data Engineer

    Production pipelines, data modelling and operational monitoring.

  4. What's next

    Open to opportunities.

    Data engineering, analytics engineering and data platform teams.

05 / Tools in context

Experience behind the stack.

Production / FanDuel

Data engineering

Python · SQL · PySpark · Databricks · Airflow · ETL/ELT · Data modelling · Validation & monitoring

Academic / Research

Machine learning

pandas · NumPy · scikit-learn · Gymnasium · Stable-Baselines3 · PPO · Model evaluation

Academic / Projects

Cloud & security

AWS EC2 & RDS · MariaDB · Raspberry Pi · Network analysis · Wireshark · Nmap · Vulnerability assessment

Open to work

Let's build
something useful.

Discuss a data engineering role, a technical project,
or a problem worth exploring.

LinkedIn ↗GitHub ↗Back to top ↑