Between the two slaves (upstream ports as they are known in the spec) you need a device capable of acting as a host for each one. There are a number of hosts out there.
Cypress makes a nice one - CY7C67200/CY7C67300
formatting link
OTG is fairly mature. You would have to replace your slaves (well, at least one of them) with an OTG device. If that's not possible, then use the multi host device.
Note that Philips makes hosts as well (ISP1362, for example), but I (and others) have had trouble making it work - seems to be the software stack, but we're really not sure. The ISP1362 has a host and a OTG port.
Cheers
PeteS