This error is indicating that you need to set another language encoding, as the default that you are using (latin-1) cannot encode some special characters (in the citations specifically! So nothing to do with RESCRIPt itself). See this topic for troubleshooting steps:
(you can also search the forum for "codec can't encode character" to see quite a few similar topics)