Out of Bounds
Out of Bounds (abbreviated OoB) is a rule applied to Super Metroid categories, typically preventing the use of Out of Bounds (e.g. no OoB) in a run.
Out of Bounds is defined as whenever Samus' hitbox leaves the regular boundaries of the room (via X-Ray Climb, Doorskip, or any other glitch) and causes the map to shift into a black area that is not marked in the game and is not tracked. The map on the HUD consequently stops blinking when this occurs (which it does not when Samus is located at black inbounds map square tiles such as the easiest to reach black map square tile in the Green Brinstar Fireflea Room). In Out of Bounds Samus completely leaves the existing room and is able to exploit junk data left behind from previous rooms.
This does not cover clipping through walls and platforms, as well as an X-Ray Climb that goes through the wall but doesn't leave the room (such as leaving Draygon without Gravity).
Kejardon's explanation of out of room travel
Samus is at any point in time during game-play (i.e. outside of where ever the game puts Samus during Intro or Title Screen menuing phases) only allowed to exit the inbounds rectangles of rooms with parts of her bodybox (or her whole bodybox) during standard/default elevator rides (i.e. non-loose ones activated by triggering the yellow elevator platform enemy that continually teleports Samus onto it while it keeps moving, whereas a loose elevator ride can be triggered using Reserves, for example), and while a transitioning process is already going on (since in some cases of entering normal doors normally, the game shifts Samus' bodybox through and past the transition tiles).
For more information also see this post http://tasvideos.org/forum/viewtopic.php?p=433442&highlight=#433442 and the bottom part of this post: http://tasvideos.org/forum/viewtopic.php?p=418314&highlight=#418314
Origin Effects
When Samus' X position is close to 0, she will have strange properties with the blocks she's touching. This is because of the collision oscillator, and the fact that she's touching blocks at both positive and negative coordinates. Every second frame will have a solid collision, and every other frame will have a non-solid collision.
Sometimes a door transition can even be encountered here without a door block, as seen in the 3% tutorial; this will be one of the non-solid collisions, and the current best guess is that this is due to the room's repeating geometry, so this collision happens against a block that's at the far right side of the room.
Practical Guide
While most categories ban the use of OoB, there are categories that allow it, and take advantage of OoB movement, such as Any% Glitched, Any% GT Code, and Low% Glitched categories (0%, 4%, and 12%).
The layout of the Out of Bounds area can be typically normalized by resetting, or by entering a larger room than the one you plan on going Out of Bounds in.
There are various types of blocks that Samus will interact with while she's moving out of bounds, including door transition blocks, spike blocks, blocks that will crash the game, and slope blocks. Less commonly, there are blocks that allow for other functions like memory corruption.
Slope blocks have strange properties out of bounds, and may cause Samus to fall partially through them to the block below, or to "bounce" on them.
Use of the X-Ray Scope once you're far enough away from the original room can change memory values, and can change the room geometry.
Mapping Out of Bounds
To figure out a particular out of bounds route, you will typically want to use a Hitbox Viewer, and figure out a combination of visual cues, audio cues, rhythmic cues such as a metronome or the Power Beam auto-fire, and Normalized Movement, to navigate where you need to go.
Normally lsnes and Snes9x use fixed data to initialize ram at startup (as does sd2snes), but a real console+cartridge might have some data left over from a previous boot, due to the nature of DRAM. It will also contain different values between individual examples of consoles, since each DRAM cell will have a specific value that it will trend towards when it's powered off.
If we instead set lsnes to randomize the ram data on startup, that gives us a bit more of a worst case scenario, showing the areas of Out of Bounds that might contain unexpected garbage blocks during actual runs.
higan and bsnes have a different RAM initialization method that partially randomizes the data in a way that's designed to more closely resemble actual console behavior. There's no hitbox viewer available for these emulators, but you can either port the initialization method into lsnes, or you can use Cheat Engine to paste the higan tilemap RAM data into lsnes' memory.
For instance, this video shows a comparison of the last OOB navigation in 0%, using the lsnes initialization pattern, as well as four patterns taken from higan: