Offline Deep Reinforcement Learning-Based Home Energy Management Systems With Heterogeneous EV Charging Load Models
Bibliographic record
Abstract
With increasing penetration of Electric Vehicles (EVs) into the transportation system and smart electricity grid, there is a growing need for integrating them into Home Energy Management Systems (HEMS). This integration within HEMS introduces dynamic user behaviors and time-varying charging demand, thus posing challenges for the HEMS. To mitigate these challenges, this paper proposes a charging model for heterogeneous EVs that covers the range of Plug-in Hybrid EVs (PHEVs), Range-Extender EVs (REEVs) and Battery EVs (BEVs) with/without heat pumps. The proposed heterogeneous EV charging model considers weather conditions, estimated mileage and driver’s experience to describe the dynamic charging demand and the anxiety level influencing their behavior. To optimize the HEMS operation, minimizing the energy cost and ensuring comfort, this paper introduces an offline Deep Reinforcement Learning (DRL) algorithm which learns directly from pre-collected datasets, avoiding the cost and safety issues associated with continuous real-world interactions. The algorithm incorporates the Huber loss and a Q-quantile estimator to mitigate performance degradation from dataset anomalies such as data noise, sensor failure and human error, resulting in more robust HEMS optimization strategies. Experimental results demonstrate the method’s effectiveness in reducing total costs and analyze the performance of household devices with two different electricity rates.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.001 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".