The initial location of the invasion is not likely to be ideal for the introduced species. Eventually it spreads to an area it is better adapted to and then begins it's growth phase.
Yes, that’s one of the environmental reasons for a lag, often people provide the dispersal vector to bring it to the favorable habitat. Sometimes, people bring the habitat to the invader – pollution abatement programs sometimes come with a blossom of colonizing species. As you’ll see I’m more interested in some much longer-term phenomena, where these issues of environmental factors will also include cultural changes.