It would be hard to do. I ended up removing the rear wheel and peeling back the liner. There are hex screws that are inside down the sides of the diffuser that are really hard to get too. I ended up going and buying a air ratchet, if you don't have one, I would recomned getting it, it will make the job so much easier and faster. There really is very little room to maneauvre inside the diffuser.
I was installing the ACP one so the bottom comes back a lot further which made it harder, if you aren't installng one of those then it can be done without removing wheels with an air ratchet.
Depending on how big you are would depend if you need to jack it up at the rear or not, but thats not so hard to do.