{"id":"W4415903164","doi":"10.24963/kr.2025/55","title":"Pushdown Reward Machines for Reinforcement Learning","year":2025,"lang":"en","type":"article","venue":"","topic":"Machine Learning and Algorithms","field":"Computer Science","cited_by":0,"is_retracted":false,"has_abstract":true,"ca_institutions":"Vector Institute; University of Toronto","funders":"Natural Sciences and Engineering Research Council of Canada; Government of Canada; Canadian Institute for Advanced Research","keywords":"Reinforcement learning; Exploit; Constant (computer programming); Pushdown automaton; Finite-state machine; Extension (predicate logic); Counterfactual thinking; Task (project management); Automaton","routes":{"ca_aff":true,"ca_fund":true,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"codex-gemma-dda1882f352a","candidate_categories":[],"consensus_categories":[],"category_scores_codex":[0.0003164482,0.00009901111,0.000114669,0.0001061392,0.00022437,0.0001448224,0.0004737005,0.0000325163,0.00003956738],"category_scores_gemma":[0.0001360082,0.0000790165,0.00007202035,0.0002432857,0.00001190408,0.0001314823,0.0001964265,0.0001407094,0.00002909816],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.00001812653,"about_ca_system_score_gemma":0.00004007018,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.00007390305,"about_ca_topic_score_gemma":0.000003297192,"domain_scores_codex":[0.9992118,0.00003146468,0.0001639356,0.0002603898,0.0001136879,0.0002187345],"domain_scores_gemma":[0.9994782,0.0001187434,0.00004162393,0.0002686814,0.00005338834,0.000039343],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"theoretical_or_conceptual","study_design_gemma":"simulation_or_modeling","study_design_scores_codex":[0.000008291312,0.00002096049,0.001657006,0.00005094499,0.00003286068,0.000001745926,0.0002207622,0.0261257,0.0001783348,0.5034565,0.01199057,0.4562563],"study_design_scores_gemma":[0.0003034318,0.0000710693,0.0003015471,0.00001642245,0.000003541111,0.000001294564,0.000009474797,0.7970735,0.0004097265,0.004702569,0.1970079,0.00009953499],"study_design_candidate":"simulation_or_modeling","study_design_consensus":null,"genre_codex":"methods","genre_gemma":"empirical","genre_scores_codex":[0.0003166677,0.00005871001,0.9165126,0.004159831,0.0004111826,0.0001365382,1.133535e-7,0.0003575048,0.07804682],"genre_scores_gemma":[0.4393093,0.00001469841,0.2488236,0.002350176,0.0001651517,0.00008755691,0.00001022631,0.00001135532,0.3092279],"genre_candidate":"methods","genre_consensus":null,"teacher_disagreement_score":0.7709478,"threshold_uncertainty_score":0.3222199,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.007398034949888881,"score_gpt":0.2720153161221875,"score_spread":0.2646172811722987,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}