The Entity Access Groups module allows site-builders to restrict entity access operations (view, create, update, delete) based on an entity relationship where the same entity bundles are referenced on both the content and user entity.

The module was built so the content entity doesn't require any field reconstruction, so the module works with existing entity references that have already been defined. You'll need to create an Entity Access Group and select the references to use for both the content and user entities.