The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.

pomdp 1.2.3 (05/04/2024)

Bugfixes

• Fixed possible memory violation in observation_matrix() and transition_matrix().

pomdp 1.2.1 (04/08/2024)

New Features

• read_POMDP gained parameter verbose to debug reading.
• solve_x check now that the model is of type x.
• Added some POMDP file examples.

Bugfixes

• Improved read_POMDP and write_POMDP.
• old LaTeX version on the CRAN master cannot deal with underscores in filenames. Rename the files cliff_walking_gridworld.png and
  windy_gridworld.png

pomdp 1.2.0 (04/02/2024)

New Features

• Added functions to work with MDP policies (see ? MDP_policy_functions).
• Added MDP solver functions: Q-learning, Sarsa, and expected Sarsa.
• simulate_MDP() and simulate_POMDP() gained parameter return_trajectories.
• New functions absorbing_states() and reachable_states() for MDPs and POMDPs.
• Support for gridworlds (see ? gridworld).
• New datasets: Cliff_walking, Windy_gridworld, RussianTiger
• plot_transition_graph() now hides unavailable actions.
• Added actions() to find available actions (unavailable actions have a reward of -Inf).
• Added make_partially_observable() and make_fully_observable() to convert between MDPs and POMDPs.

Changes

• simulate_POMDP(): Better calculation of T for infinite-horizon problems.
• several functions are now generics with methods for POMDP and MDP.
• policy() lost the parameters alpha and action.
• policy() and value_function() and gained the parameter drop.
• regret(): renamed parameter belief to start. Regret is now available for MDPs.
• simulate_MDP() stops now at absorbing states.
• simulate_MDP_cpp() works now with sparse model representation.
• POMDP and MDP gained field for additional info.
• approx_MDP_policy_evaluation() is now called MDP_policy_evaluation() and gained parameter theta as an additional stopping criterion.
• rewrote all accessor code reward_matrix, transition_matrix, observation_matrix for better and faster access.
• normalize() gained parameters for more detailed normalization.
• POMDP() and MDP() lost normalize.
• model.h has now support for keywords in transition_prob and observation_prob.
• MDP2POMDP is now make_partially_observable().

Bugfixes

• q_values_MDP(), solve_MDP(): Fixed reward representation issue.
• reward_val_cpp(): fixed observation matching bug.

pomdp 1.1.3 (12/20/2023)

New Features

• simulate_POMDP() and simulate_MDP() gained parameter delta_horizon and calculates now the horizon for infinite-horizon problems.
• added add_policy() and several consistency checks.

Changes

• Changed the action names for the Maze example to the names used in Russell and Norvig’s AIMA book.
• POMDP lost the parameter max. Costs need to be specified as negative rewards.

Bugfixes

• simulate_POMDP() now adds terminal values.

pomdp 1.1.2 (09/07/2023)

Bugfixes

• Fixed memory access bug in model.h

pomdp 1.1.1 (09/04/2023)

Changes

• plot_policy_graph(): The parameter order has slightly changed; belief_col is now called state_col; unreachable states are now suppressed.
• policy() gained parameters alpha and action.
• color palettes are now exported.
• POMPD accessors gain parameter drop.
• POMDP constructor and read_POMDP gained parameter normalize and, by default, normalize the POMDP definition.

New Features

• Large POMDP descriptions are now handled better by keeping the reward as a data.frame and supporting sparse matrices in the C++ code.
• New function value_function() to access alpha vectors.
• New function regret() to calculate the regret of a policy.
• transition_graph() to visualize the transition model.

pomdp 1.1.0 (01/23/2023)

New Features

• Added C++ (Rcpp) support. Speed up for simulate_POMDP, sample_belief_space, reward, …
• simulate_POMDP and sample_belief_space now have parallel (foreach) support.
• Sparse matrices from package Matrix for matrices with a density below 50%.
• Added support to parse matrices for POMDP files.
• Added model normalization.
• is_solved_POMDP(), is_converged_POMDP(), is_timedependent_POMDP(), and is_solved_MDP() are now exported.

Changes

• accessors are now called now transition_val() and observation_val().
• simulate_POMDP() and simulate_MDP() now return a list.
• reimplemented round_stochastic() to improve speed.
• MDP policy now uses factors for actions.
• estimate_belief_for_nodes() now can also use trajectories to estimate beliefs faster.
• cleaned up the interface for episodes and epochs.

pomdp 1.0.3 (05/18/2022)

• Fixed rounding issue on some architectures.

pomdp 1.0.2 (05/17/2022)

• policy_graph() can now produce policy trees for finite-horizon problems and the initial belief can be specified.
• simulate_POMDP(): fixed bug with not using horizon.
• reward() and reward_node_action() have now been separated.
• sample_belief_space() gained method ‘trajectories’.
• simulate_POMDP(): supports not epsilon-greedy policies.
• added x_prob() and x_val() functions to access individual parts of the matrices.
• fixed converged finite-horizon case. It now only returns the converged graph/alpha.
• we use not internally NA to represent * in the POMDP definition.
• actions, states and observations are now factors in most places.

pomdp 1.0.1 (03/25/2022)

• Fixed rounding issue on some architectures.
• Fixed bug in write_POMDP() (reported by emile-pelletier-gc).
• estimate_belief_for_nodes() is now exposed and the code has been improved.

pomdp 1.0.0 (02/23/2022)

• POMDP objects now have no list element model, but are the model list directly.
• moved pomdp-solve to package pomdpSolve.
• added solve_MDP().
• transition probability, observation probabilities and rewards can now be specified as a function.
• transition_matrix et al now can also return a function.
• Improved POMDP file writer.

pomdp 0.99.3 (08/05/2021)

• moved Ternary and visNetwork to SUGGESTED.
• removed clang warning for lex scanners.

pomdp 0.99.2 (05/14/2021)

Bugfix

• Removed nonportable flag -C from Makefile.

pomdp 0.99.1 (05/13/2021)

New Features

• Added a wrapper for the sarsop library.

Changes

• Improved error messages when accessing fields not parsed by read_POMDP.
• policy() no longer returns the graph, but just alphas and the optimal action.
• The maintainer is now mhahsler.

Bugfix

• Resolved issues with factors for R 4.0. We now mostly use character instead of factors.
• States and actions as numbers are now handled correctly (reported by meeheal).
• Added spelling fixes by brianrice2.
• Fixed buffer overflow for filename parameters in pomdpsolve.

pomdp 0.99.0 (05/04/2020)

Changes

• Support finite-horizon POMDPs and store epochs.
• reward now looks at different epochs, calculates the optimal actions and the parameter names are improved.
• solve_POMDP not looks at convergence.
• solve_POMDP gained parameter terminal_values.
• solve_POMDP gained parameter discount to overwrite the discount rate specified in the model.
• solve_POMDP can now solve POMDPs with time-dependent transition probabilities, observation probabilities and reward structure.
• solve_POMDP gained parameter grid in parameter list to specify a custom belief point grid for the grid method.
• write_POMDP and solve_POMDP gained parameter digits.
• added read_POMDP to read POMDP files.
• plot for POMDP is now replaced by plot_policy_graph.
• added policy graph visualization with visNetwork.
• added plot_value_function.
• added function sample_belief_space to sample from the belief space.
• added function plot_belief_space.
• added function transition_matrix.
• added function observation_matrix.
• added function reward_matrix.
• POMDP model now also contains horizon and terminal_values.
• added MDP formulated as a POMDP.
• added policy function to extract a better readable policy.
• added update_belief.
• added simulate_POMDP.
• added round_stochastic.
• added optimal_action.

pomdp 0.9.2 (12/06/2019)

Changes

• solve_POMDP can now solve POMDP files.
• added helper functions O, R and T.
• improved plot.
• Added reward function.
• values argument is now called max.
• Fixed class structure. The central class is not POMDP with elements model and solution.

Bugfix

• fixed warning for start = “uniform”.
• fixed warning in C code for gcc10.

pomdp 0.9.1-1 (05/14/2019)

Bugfix

• fixed warning in mdp.c for gcc9.

pomdp 0.9.1 (01/02/2019)

Bugfix

• Fixed Warning in fg-params.c

New Features

• New method transitions to extract the transition matrix from a POMDP.

pomdp 0.9.0 (12/25/2018)

Initial CRAN release.

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.