When you test the board, you need to check "include FPGA Board in the test" checkbox to do a full test. Otherwise, it will not try to run the simulation.
When you get error message "Failed to initialize the RTIOStream library" in simulation, can you tell if there's error message printed at the MATLAB console at the same time? Sometimes the error message is printed there.
It would be helpful if you can provide more information regarding your FPGA board.