You need to add the objects you want to be effected into a collection then add that collection to the modifier.