Highlight values not between X and Y

When you use a formula to apply conditional formatting, the formula is evaluated for each cell in the range, relative to the active cell in the selection at the time the rule is created. So, in this case, if you apply the rule to B4:G11, with B4 as the active cell, the rule is evaluated for each of the 40 cells in B4:G11 because B4 is entered as a fully relative address. Because we are using OR with two conditions, the formula returns TRUE when either condition return TRUE, triggering the conditional formatting.

Using other cells as inputs

You don't have to hard-code the numbers into the rule and, if the numbers will change, it's better if you don't.

To make a more flexible, interactive conditional formatting rule, use other cells like variables in the formula. For example, if you want to use cell E2 for the lower limit, and cell G2 for the upper limit, you can use this formula:

=AND(B4<$E$2,A1>$G$2)

You can then change the values in cells E2 and G2 to anything you like and the conditional formatting rule will respond instantly. You must use an absolute address for E2 and G2 to prevent these addresses from changing.

With named ranges

A better way to lock these references is to use a named ranges, since named ranges are automatically absolute. If you name cell E2 "lower" and the cell G2 "upper", then you can write the conditional formatting formula like so:

=AND(B4upper)

Named ranges allow you to use a cleaner, more intuitive syntax.