Let me say first off that this isn't the most elegant of ways to accomplish it. It is in the "it works for me" stage.
A quick primer on EXE::Custom: This is a setting just like RHOST in Metasploit wherever an EXE is built for Windows payloads. Such as PSEXEC, BypassUAC, etc. It tells Metasploit to ignore all of your payload settings and just use the EXE you have specified. Now this does come at a bit of a cost. If you don't have a handler set up to catch whatever is in your EXE and you have bogus information in your payload settings for your module, Metasploit won't be able to magically determine what you want it to do. So make sure that you either have the payload settings correct (even though it won't use them for the exploit) or set the option "DisablePayloadHandler" to true and start your own handler somewhere else.
With that said, the Java Signed Applet attack is one of the most widely used, one problem in the Metasploit module version is that you have no external control (such as EXE::Custom) for the binary that is dropped if you are using a Windows native payload. (I do highly recommend trying out Java Meterpreter, but that it for another post)
Here is the hack that gets me by until a more elegant solution is thought up by smarter people than I:
We are going to use the Java Rhino exploit, and this story starts with an undetectable.exe sitting in /tmp/ - How you bypass the AV you are up against is up to you. The first thing we need to do is make a core library change. Yes, this is as scary as it sounds and you should document any time you do so you can revert as needed.
In your MSF directory open up lib/msf/util/exe.rb - should look something like this:
Scroll down or search for "self.to_jar" - should look something like this:
Now add "exe = File.read('/path/to/your/evil.exe') as demonstrated below:
Cool. We've made our change, now we have to pull down a jar generated with our evil bin, and the Rhino exploit. We can do that by spinning up Metasploit with our newly editing core library. Load up the Rhino exploit via "use exploit/multi/browser/java_rhino".
One trip up that I messed up even just creating this blog post (and a constant headache in the #Metasploit channel on Freenode) is forgetting to set the TARGET variable in Java exploits. The default payload type being Java. Since we are using a native windows binary we need to specify Windows by setting TARGET 1.
Run the exploit with any settings, since we'll be just pulling out what we need. Then pull down the exploit JAR file with wget:
(Most java exploits are set to answer ANY .jar extension request with the payload.)
The HTML to load the JAR is pretty straight forward, but if you've never seen it before here it is:
Thats it. A very blank page with an exploit in it. We can do better than that:
Obviously you can do whatever you want with the HTML, mirror a site or what have you. Notice that I've also changed the name of the JAR. (You can't change the class unless you start making changes to the exploit directly and I wanted to keep this as generically applicable as possible)
Move the Applet.jar to SuperMario.jar and start up a Web server (for demo using Python's awesome SimpleHTTPServer, but apache will serve as well)
Don't forget to start up a handler for our now-external exploit:
Victim views the site:
And WA-LA! we haz shell:
Yay… Game over..