Depends on a lot of factors. Manual collections will "refresh" all flood playouts (those that use schedules) whenever they are modified, so you will notice changes immediately. Smart collections will be re-evaluated every time the playout is extended, meaning new items will work their way in over time.
Whether these changes affect progress through the collection depends on the playback order you have selected. Shuffled collections will shuffle differently when they have more items, so you may see repeated content sooner if you regularly modify a shuffled collection. Chronological collections should be better at maintaining progress through a collection, though it may be off by up to the number of items added to the collection, depending on if they sort before or after the current position in the collection.
With block schedules and YAML playouts, chronological should better at maintaining progress in almost all cases, and they use much more accurate history to remember the last item that was played, and will find it or the closest item in the collection before extending the playout.