is there a way to inject my custom driver instance into SeleniumLibrary instead using its own driver.
Example: I have a remote webdriver driver instance I want to inject into SeleniumLibrary and later use selenium library actions like click etc
I think 
that what you are asking is something like Selenium Grid, where you point your tests to the browsers in a service. In your case you would be the creator of that service, and consumer, of course. See what docs they have at Selenium, then it is a matter to adjust to SeleniumLibrary.