It depends. The butterfly effect itself is just a theory, not a law. There is no way to completely tell if one small, insignificant event causes or prevented a huge event from happening, though its unlikely.
It migjt matter in cases where the moment is instant. For example, leaving a minute late might make it so you avoid a major accident, but that doesn't mean if you left at the right time you would've been hit.
In the end, little things just might cause or prevent major events, but there is no way to tell for sure. There are always what its and uncertainties