upkie.envs.upkie_base_env.UpkieBaseEnv Class Reference

Base class for Upkie environments. More...

Public Member Functions
None	__init__ (self, float fall_pitch=1.0, Optional[float] frequency=200.0, bool frequency_checks=True, Optional[RobotState] init_state=None, bool regulate_frequency=True, str shm_name="/upkie", Optional[dict] spine_config=None, int spine_retries=10)
	Initialize environment. More...
def	__del__ (self)
	Stop the spine when deleting the environment instance.
None	close (self)
	Stop the spine properly.
Optional[float]	dt (self)
	Regulated period of the control loop in seconds, or None if there is no loop frequency regulation.
Optional[float]	frequency (self)
	Regulated frequency of the control loop in Hz, or None if there is no loop frequency regulation.
None	update_init_rand (self, **kwargs)
	Update initial-state randomization. More...
Tuple[np.ndarray, dict]	reset (self, *Optional[int] seed=None, Optional[dict] options=None)
	Resets the spine and get an initial observation. More...
Tuple[np.ndarray, float, bool, bool, dict]	step (self, np.ndarray action)
	Run one timestep of the environment's dynamics. More...
bool	detect_fall (self, dict spine_observation)
	Detect a fall based on the base-to-world pitch angle. More...
def	get_env_observation (self, dict spine_observation)
	Extract environment observation from spine observation dictionary. More...
dict	get_spine_action (self, action)
	Convert environment action to a spine action dictionary. More...
None	log (self, str name, Any entry)
	Log a new entry to the "log" key of the action dictionary. More...
None	bullet_extra (self, dict bullet_action)
	Prepend for the next step an extra action for the Bullet spine. More...

Public Attributes
	fall_pitch
	Fall detection pitch angle, in radians.
	init_state
	Initial state for the floating base of the robot, which may be randomized upon resets.
	model
	Robot model read from its URDF description.

Detailed Description

Base class for Upkie environments.

This environment base class provides a number of features:

• Communication with the spine process.
• Fall detection.
• Initial state randomization (e.g. when training a policy).
• Loop frequency regulation (optional).

It otherwise defines three abstract methods to be defined by derived environments to:

• Convert environment action to spine action.
• Convert spine to environment observations.
• Get reward for the current observation and action.

Note

Upkie environments are made to run on a single CPU thread. The downside for reinforcement learning is that computations are not massively parallel. The upside is that it simplifies deployment to the real robot, as it relies on the same spine interface that runs on Upkie.

Constructor & Destructor Documentation

__init__()

None upkie.envs.upkie_base_env.UpkieBaseEnv.__init__	(		self,
		float	fall_pitch = 1.0,
		Optional[float]	frequency = 200.0,
		bool	frequency_checks = True,
		Optional[RobotState]	init_state = None,
		bool	regulate_frequency = True,
		str	shm_name = "/upkie",
		Optional[dict]	spine_config = None,
		int	spine_retries = 10
	)

Initialize environment.

Parameters

fall_pitch	Fall detection pitch angle, in radians.
frequency	Regulated frequency of the control loop, in Hz. Can be prescribed even when regulate_frequency is unset, in which case self.dt will be defined but the loop frequency will not be regulated.
frequency_checks	If regulate_frequency is set and this parameter is true (default), a warning is issued every time the control loop runs slower than the desired frequency. Set this parameter to false to disable these warnings.
init_state	Initial state of the robot, only used in simulation.
regulate_frequency	If set (default), the environment will regulate the control loop frequency to the value prescribed in frequency.
shm_name	Name of shared-memory file to exchange with the spine.
spine_config	Additional spine configuration overriding the default upkie.config.SPINE_CONFIG. The combined configuration dictionary is sent to the spine at every reset.
spine_retries	Number of times to try opening the shared-memory file to communicate with the spine.

Exceptions

SpineError

If the spine did not respond after the prescribed number of trials.

Reimplemented in upkie.envs.upkie_ground_velocity.UpkieGroundVelocity, upkie.envs.upkie_servo_positions.UpkieServoPositions, upkie.envs.upkie_servo_torques.UpkieServoTorques, and upkie.envs.upkie_servos.UpkieServos.

Member Function Documentation

bullet_extra()

None upkie.envs.upkie_base_env.UpkieBaseEnv.bullet_extra	(		self,
		dict	bullet_action
	)

Prepend for the next step an extra action for the Bullet spine.

This extra action can be for instance a set of external forces applied to some robot bodies.

Parameters

bullet_action

Action dictionary processed by the Bullet spine.

detect_fall()

bool upkie.envs.upkie_base_env.UpkieBaseEnv.detect_fall	(		self,
		dict	spine_observation
	)

Detect a fall based on the base-to-world pitch angle.

Parameters

spine_observation

Observation dictionary with a "base_orientation" key. This requires the upkie::cpp::observers::BaseOrientation observer in the spine's observer pipeline.

Returns

True if and only if a fall is detected.

get_env_observation()

def upkie.envs.upkie_base_env.UpkieBaseEnv.get_env_observation	(		self,
		dict	spine_observation
	)

Extract environment observation from spine observation dictionary.

Parameters

spine_observation

Spine observation dictionary.

Returns

Environment observation.

Reimplemented in upkie.envs.upkie_ground_velocity.UpkieGroundVelocity, and upkie.envs.upkie_servos.UpkieServos.

get_spine_action()

dict upkie.envs.upkie_base_env.UpkieBaseEnv.get_spine_action	(		self,
			action
	)

Convert environment action to a spine action dictionary.

Parameters

action

Environment action.

Returns

Spine action dictionary.

Reimplemented in upkie.envs.upkie_servos.UpkieServos, and upkie.envs.upkie_ground_velocity.UpkieGroundVelocity.

log()

None upkie.envs.upkie_base_env.UpkieBaseEnv.log	(		self,
		str	name,
		Any	entry
	)

Log a new entry to the "log" key of the action dictionary.

Parameters

name	Name of the entry.
entry	Dictionary to log along with the actual action.

reset()

Tuple[np.ndarray, dict] upkie.envs.upkie_base_env.UpkieBaseEnv.reset	(		self,
		*Optional[int]	seed = None,
		Optional[dict]	options = None
	)

Resets the spine and get an initial observation.

Parameters

seed	Number used to initialize the environment’s internal random number generator.
options	Currently unused.

Returns

• observation: Initial vectorized observation, i.e. an element of the environment's observation_space.
• info: Dictionary with auxiliary diagnostic information. For Upkie this is the full observation dictionary sent by the spine.

Reimplemented in upkie.envs.upkie_ground_velocity.UpkieGroundVelocity.

step()

Tuple[np.ndarray, float, bool, bool, dict] upkie.envs.upkie_base_env.UpkieBaseEnv.step	(		self,
		np.ndarray	action
	)

Run one timestep of the environment's dynamics.

When the end of the episode is reached, you are responsible for calling reset() to reset the environment's state.

Parameters

action

Action from the agent.

Returns

• observation: Observation of the environment, i.e. an element of its observation_space.
• reward: Reward returned after taking the action.
• terminated: Whether the agent reached a terminal state, which may be a good or a bad thing. When true, the user needs to call reset().
• truncated: Whether the episode is reaching max number of steps. This boolean can signal a premature end of the episode, i.e. before a terminal state is reached. When true, the user needs to call reset().
• info: Dictionary with auxiliary diagnostic information. For us this is the full observation dictionary coming from the spine.

update_init_rand()

None upkie.envs.upkie_base_env.UpkieBaseEnv.update_init_rand	(		self,
		**	kwargs
	)

Update initial-state randomization.

Keyword arguments are forwarded as is to upkie.utils.robot_state_randomization.RobotStateRandomization.update.

---

The documentation for this class was generated from the following file:

• upkie/envs/upkie_base_env.py