Markov Decision Processes (MDPs)

Documentation for package ‘MDP2’ version 3.0.0

Help Pages

binary_action_writer	Function for writing actions of a HMDP model to binary files. The function defines sub-functions which can be used to define actions saved in a set of binary files. It is assumed that the states have been defined using 'binary_mdp_writer' and that the id of the states is known (can be retrieved using e.g. 'state_idx_df').
binary_mdp_writer	Function for writing an HMDP model to binary files. The function defines sub-functions which can be used to define an HMDP model saved in a set of binary files.
convert_binary_to_hmp	Convert a HMDP model stored in binary format to a 'hmp' (XML) file. The function simply parse the binary files and create 'hmp' files using the 'hmp_mdp_writer()'.
convert_hmp_to_binary	Convert a HMDP model stored in a hmp (xml) file to binary file format.
get_bin_info_actions	Info about the actions in the HMDP model under consideration.
get_bin_info_states	Info about the states in the binary files of the HMDP model under consideration.
get_hypergraph	Return the (parts of) state-expanded hypergraph
get_info	Information about the MDP
get_policy	Get parts of the optimal policy.
get_rpo	Calculate the retention pay-off (RPO) or opportunity cost for some states.
get_steady_state_pr	Calculate the steady state transition probabilities for the founder process (level 0).
get_w_idx	Return the index of a weight in the model. Note that index always start from zero (C++ style), i.e. the first weight, the first state at a stage etc has index 0.
hmp_mdp_writer	Function for writing an HMDP model to a hmp file (XML). The function define sub-functions which can be used to define an HMDP model stored in a hmp file.
load_mdp	Load the HMDP model defined in the binary files. The model are created in memory using the external C++ library.
memory_mdp_writer	Function for building an HMDP model directly in memory.
plot.HMDP	Plot the state-expanded hypergraph of the MDP.
plot_hypergraph	Plot parts of the state expanded hypergraph.
random_hmdp	Generate a "random" HMDP stored in a set of binary files.
run_calc_weights	Calculate weights based on current policy. Normally run after an optimal policy has been found.
run_policy_ite_ave	Perform policy iteration using the average expected-weight Bellman operator on the MDP.
run_policy_ite_discount	Perform policy iteration using the discounted expected-weight Bellman operator on the MDP.
run_value_ite	Perform value iteration on the MDP.
save_mdp	Save the MDP to binary files
set_policy	Modify the current policy by setting policy action of states.