This could be because
- User does not has repository access.
- User has a different username in Bitbucket and IDP.
The Test Configuration screen only verifies if the plugin configuration is correct and checks connection with the configured provider.
It does not check your access rights over any of the repositories and whether the user is present in the local Bitbucket directory.