Sprite locator: Multiple sprites in a single frame


Hi there!

I’m just generally messing around with serpent, working on a Super Mario game agent. Yesterday I took the time to rip some sprites other than Mario, and I noticed that I always only get a single hit no matter how many sprites are present in the frame. I checked the code for the locate() function and it seems designed to only return a single hit in each frame. Do you have any suggestions for locating multiple sprites of the same class in a single game frame? I have a few ideas, but I wanted some input first. Thanks!


I had an idea yesterday that I implemented, and it seems to work pretty well, albeit a little bit slow. I guess the SpriteLocator isn’t really meant to do real time tracking of lots of things at once on a full scale image (512 x 448), who could have guessed it might be slow :stuck_out_tongue: My solution is to blot out the found sprite with black pixels and run the locator again.

        def find_all_enemies(cls, game_frame: GameFrame):
            frame = game_frame.frame
            api = MarioASAPI.instance
            sprites = api.game.sprites
            results = []
            while True:
                result = api.sprite_locator.locate(sprite=sprites['SPRITE_GOOMBA'], game_frame=GameFrame(frame))
                if result is None:
                frame[result[0]:result[2], result[1]:result[3]] = (0, 0, 0)
                print(f'found: {result}  num_hits: {len(results)}')

            return None if not results else results

It kind of works for some purposes though. This will naturally break if sprites are overlapping, which will essentially have the same result as one sprite being slightly outside of the screen, but I’m not sure the locator algorithm is meant for that type of robustness.