End-to-End Learning for Partially-Observed Time Series with PyPOTS
📌 Abstract & Overview
Partially-observed time series (POTS) are ubiquitous in real-world applications (IoT sensors, healthcare vitals, industrial monitors, and financial records). However, conventional machine learning toolchains isolate missing-value imputation from downstream predictive modeling, causing error propagation, poor reusability, and fragmented codebases.
This KDD 2026 Hands-on Tutorial introduces PyPOTS, an open-source Python ecosystem for end-to-end data mining and machine learning on POTS. We present unified workflows covering missingness simulation, data preprocessing, neural modeling, and evaluation across five core tasks: Imputation, Forecasting, Classification, Clustering, and Anomaly Detection.
🎯 Learning Outcomes & Target Audience
Learning Outcomes
By participating in this 3-hour interactive tutorial, attendees will learn how to:
- Deploy PyPOTS to construct automated pipelines for time series with arbitrary missingness mechanisms (MCAR, MAR, MNAR).
- Benchmark SOTA Models using PyPOTS’s unified APIs across multiple machine learning tasks.
- Extend PyPOTS Architecture by writing custom neural network backbones, cores, and loss objectives.
- Follow Open-Source Best Practices to contribute custom algorithms, unit tests, and extensions back to PyPOTS.
Target Audience & Prerequisites
- Practitioners, Data Scientists, & Software Engineers: Working with incomplete sensor data and time-series pipelines.
- Researchers: Developing novel deep learning models for time-series representation and imputation.
- Prerequisites: Basic knowledge of Python, Jupyter Notebooks, and PyTorch basics.
👨🏫 Tutors & Presenters Biography
Creator and R&D lead of PyPOTS. Focuses on AI systems for time series analysis and real-world ML development.
Research focuses on intelligent sensing and spatio-temporal data mining. Previously at Tsinghua, Microsoft, Alibaba, and Huawei.
Research interests include time series analysis, information theory, evidence theory, and complex physical systems.
Published 200+ papers in top venues (KDD, NeurIPS, ICML). Chairs IEEE CIS Task Force on AI for Time Series.
Research focuses on AI for Healthcare and reliable time-series learning systems.
📚 Tutorial Outline & Curriculum
The tutorial is split into two tightly connected parts:
- I.1 Problem setup & POTS fundamentals: Missingness mechanisms (MCAR, MAR, MNAR) and evaluation principles.
- I.2 Data preparation & missingness simulation: Dataset loading, splitting, and controlled missingness injection with PyGrinder & BenchPOTS.
- I.3 Unified model training: Running SOTA models for imputation, forecasting, classification, clustering, and anomaly detection.
- I.4 Evaluation & reproducibility checklist: Metric selection, error analysis, visualization, and reproducible pipeline construction.
- II.1 PyPOTS architecture walkthrough: 3-Layer structure (backbone
nn/modules,core.py,model.py), 3 integration paths, and 6 task base classes. - II.2 Implementing a custom model/module: Step-by-step creation of a custom neural network imputer (
CustomTemporalConvImputer) from scratch. - II.3 Supporting domain constraints: Handling irregular sampling (
deltas), feature-wise masks, and custom domain loss functions (weighted MSE). - II.4 Benchmark integration & testing: Writing unit tests with PyPOTS testing standards (
tests/) and running fair baseline comparisons./ .py - II.5 Open-source workflow & contribution practice: Black code formatting, NumPy docstrings,
pypots-cli dev, and PR contribution guidelines.
📜 Citation
If you use PyPOTS or reference this tutorial in your research, please cite our papers:
@inproceedings{du2026pypotstutorial,
title={End-to-End Learning for Partially-Observed Time Series with PyPOTS},
author={Du, Wenjie and Yang, Yiyuan and Zhan, Tianxiang and Wen, Qingsong},
booktitle={Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '26)},
year={2026}
}
@article{du2023pypots,
title = {{PyPOTS: A Python Toolkit for Data Mining on Partially-Observed Time Series}},
author = {Wenjie Du},
journal = {SIGKDD MiLeTS Workshop},
year = {2023},
}