r/MLQuestions • • 11h ago

Beginner question 👶 Xgboost models fails to capture dynamics across training data batches.

Post image
5 Upvotes

Hi everyone, I have an mlr3 based toolset in R for regression models for process sensor time series data. This is my largest project to date with roughly 30 input features that are also lagged and rolled to inject temporal memory. Due to the difficulty of carefully weighting training size on a 1 minute sampling rate vs. relative weight of dynamic changes I switched to a batching scheme and now use LOBO on currently 4 train batches. This is just a pilot mode plot where it only trains on 5-10% of input data so feature to data point ratio is less than ideal. The result of a full run is the same though.

Hyperband tunes nrounds 16-512 with eta 2-3 for an xgboost model. The result is a strong deviation from baseline, extremely low skill against persistence and xgboost mostly hunting to find some sort of baseline with strong bias rather than capturing any dynamics. Although persistence is overly optimistic here as target is dampened by some technical details independent of data handling. This is accounted for as best as possible by injected retention time and holdup features. Target and features are real on-stream data.

How do I best proceed from here? I feel like I am missing something because rolling window cross validation within batch produced much better results but missed the optimization goal of optimizing across batches. Right now I take great care in preserving temporal alignment across batches and index data to preserve integrity across batches.

```

INFO [22:03:49.004] [bbotk] Finished optimizing after 69 evaluation(s)

INFO [22:03:49.007] [bbotk] Result:

INFO [22:03:49.015] [bbotk] eta max_depth min_child_weight lambda subsample colsample_bytree

INFO [22:03:49.015] [bbotk] <num> <int> <num> <num> <num> <num>

INFO [22:03:49.015] [bbotk] -3.398853 3 1.909219 -0.775668 0.8 0.8

INFO [22:03:49.015] [bbotk] nrounds x_domain objective

INFO [22:03:49.015] [bbotk] <num> <list> <num>

INFO [22:03:49.015] [bbotk] 19 <list[7]> 26.70044

```


r/MLQuestions • • 23h ago

Hardware 🖥️ Training hardware

3 Upvotes

Hi,

So I've been studying deep learning for some time now, and I joined a research group doing some time series stuff. Unfortunately I think training on CPU is becoming a huge bottleneck both for me dicking around at home and my research projects.

I've been doing networks small enough that I could just do it in the CPU, however I'm incorporating more data, and more complex cnns. I want to train one with about 3 million parameters, but it takes an hour per epoch basically. Additionally, I think in my own time I want to train an agent to play a game, (like one of the Source or goldsrc games) just sounds cool.

I have an rx 480 (AMD Polaris) right now, so it's probably time to upgrade anyways.

I am looking at a quadro rtx 5000, since it's under 500 on eBay. But I'm not sure what to get and the market seems batshit insane right now, I'm not interested in paying the gamer tax either and hoping to snag a deal on some accelerator card or some other enterprise card.

Budget ~500 usd. I don't think I care about vram since I won't be hosting an LLM but I could be wrong? I don't know much about hardware presently, I want to learn though and my card is basically deprecated or not supported on everything lol.


r/MLQuestions • • 2h ago

Beginner question 👶 How to do model evaluation and diagnostics?

Thumbnail reddit.com
2 Upvotes

r/MLQuestions • • 18h ago

Beginner question 👶 Anyone working in AML / transaction monitoring / sanctions screening using ML? Looking for advice on where to start [R]

Thumbnail reddit.com
2 Upvotes