Browsers are finally catching up; It isn’t the Future of Web Pages is it Bespin now learning some art in the Dojo
Feb 26

Supporting the system clipboard in your Web applications; What a pain!

Bespin, Tech with tags: Add comments

copypaste

When we launched Bespin people noticed that our copy and paste experience wasn’t the best. It wasn’t integrating with the system clipboard in most cases. An editor needs clipboard support. Needs it. Essential. 101.

It turns out that this can be a royal pain to work across various browsers and platforms. I thought it may be good to share our story, where we were, where we are, and where we want to be. I also am looking forward to hearing your ideas, as we may have missed something.

First, what are we talking about here?

With our canvas based editor we needed access to the system clipboard to be able to copy data into, cut, and paste from the outside clipboard.

The first version that we had, used Clipboard Copy which accesses the clipboard APIs available in Flash. Unfortunately, our parade was rained on. With Flash 10 Adobe changed the access model to these APIs (due to security issues). Now a user has to explicitly click on a SWF control to get the access. To help, zeroclipboard tricks Flash, allowing you to at least fake out the user and have them click on any element. zeroclipboard hides a translucent SWF control on top of the element so the user thinks they are clicking on the “copy” icon, or something like it. This is all well and good, but we obviously need to tie into the key combinations that we habitually know (Cmd/Ctrl C, X, and V).

With Flash not able to do what we need (as Flash 10 will be well supported soon, and it broke it) we needed to take a fresh look at the problem.

What APIs do we have available in the browsers? In typical fashion, each browser does things differently.

IE had an early API to give you access. I won’t go into detail on this puppy because we aren’t interested in supporting IE yes, until we can get Canvas/fillText running in a performant manner. Then we will care.

What about Firefox?

Firefox has a rich clipboard API in the XUL platform itself via the nsIClipboard interface. It has everything we could possibly need, and we can implement copy and paste easily:

copy: function(copytext) {
    try {
        if (netscape.security.PrivilegeManager.enablePrivilege) {
            netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect");
        } else {
            clipdata = copytext;
            return;
        }
    } catch (ex) {
        clipdata = copytext;
        return;
    }
 
    var str = Components.classes["@mozilla.org/supports-string;1"].
                              createInstance(Components.interfaces.nsISupportsString);
    str.data = copytext;
 
    var trans = Components.classes["@mozilla.org/widget/transferable;1"].
                           createInstance(Components.interfaces.nsITransferable);
    if (!trans) return false;
 
    trans.addDataFlavor("text/unicode");
    trans.setTransferData("text/unicode", str, copytext.length * 2);
 
    var clipid = Components.interfaces.nsIClipboard;
    var clip   = Components.classes["@mozilla.org/widget/clipboard;1"].getService(clipid);
    if (!clip) return false;
 
    clip.setData(trans, null, clipid.kGlobalClipboard);
},
 
data: function() {
    try {
        if (netscape.security.PrivilegeManager.enablePrivilege) {
            netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect");
        } else {
            return clipdata;
        }
    } catch (ex) {
        return clipdata;
    }
 
    var clip = Components.classes["@mozilla.org/widget/clipboard;1"].getService(Components.interfaces.nsIClipboard);
    if (!clip) return false;
 
    var trans = Components.classes["@mozilla.org/widget/transferable;1"].createInstance(Components.interfaces.nsITransferable);
    if (!trans) return false;
    trans.addDataFlavor("text/unicode");
 
    clip.getData(trans, clip.kGlobalClipboard);
 
    var str       = new Object();
    var strLength = new Object();
    var pastetext = "";
 
    trans.getTransferData("text/unicode", str, strLength);
    if (str) str = str.value.QueryInterface(Components.interfaces.nsISupportsString);
    if (str) pastetext = str.data.substring(0, strLength.value / 2);
    return pastetext;
}

Verbose isn’t it? Hardly the most succinct API ever for the common case, but it does the trick. Or does it.

You can see the real problem here:

try {
    if (netscape.security.PrivilegeManager.enablePrivilege) {
        netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect");
    } else {
        clipdata = copytext;
        return;
    }
} catch (ex) {
    clipdata = copytext;
    return;
}

For security, we have to ask the user to give us permission to use copy and paste. You get one of those ugly dialogues, and the user will be asked repeatedly for each session.

Not only is this a pretty poor user experience, but we can’t even get this far. When we ask for the priviledge, depending on the users settings, changes are that you will never actually get asked to grant them. You have to sign JavaScript and the like to even get close. This is pretty crazy. I don’t even get a CHANCE to ask the user for extended privileges (unless you get users to do magic like: user_pref("signed.applets.codebase_principal_support", true);). Here is where the cliff of the Web platform raises its ugly head. We need to fix this!

Time to get subtle here. Our use case isn’t about being able to sneakily get access to the clipboard without the user knowing. All we want it to actually get access based on the user doing something. When they Cmd/Ctrl C, X, or V. When they click on a copy button, then do the action. This is very different than getting access to the clipboard at any time.

In the code above, notice what the exception cause is doing: clipdata = copytext; This is our worst case scenario. If we can’t get access to the clipboard then we keep an internal clipboard that only allows you to cut/copy/paste within the editor itself. As soon as you want to copy something from a web page and paste it into the editor? Outta luck. This isn’t good enough.

Now, let’s take a look at WebKit. WebKit does a decent job and implementing the initial Microsoft work in a nicer DOM way.

There are a set of DOM events that have pairings that tell you before* “getting ready to do the action so set things up if you need” and then the action itself.

  • beforecopy
  • copy
  • beforecut
  • cut
  • beforepaste
  • paste

In our world with the editor, we use the before events to set things up, and we have to do something pretty hacky to make it happen. The copy event itself only actually goes through if you are on an element that supports it. There are hacks around this too. For example, if you want to be able to get a copy event on a div, you need to turn on contentEdible and set the tab index to -1. Strange huh?

To get around all of this, we use a hidden text input, which can of course accept these events. Then, in the before event we focus over to that hidden element. Here is an example for beforecopy:

Event.observe(document, "beforecopy", function(e) {
    e.preventDefault();
    $('copynpaster').focus();
});

You will notice that we make a call to preventDefault. This tells the system that we are in control and are handling the copy ourselves. This turns out to be important is some subtle ways too. For example, The OS will often grey out a menu choice for “Copy” if it doesn’t see anything selected. Since we are in control of the selection, we need to override that behavior and continue to pass through the event. In fact, I noticed a few weird things happening if you don’t get this all just right. For example, in WebKit, if you select the “Edit” main menu, it would fire a beforecopy right away, and then once again when you selected copy. Strange.

But, now we have gotten beforecopy to focus us in the text input, we are ready to implement the copy event which will not get passed through. Here we need to do OUR magic to get the selection that we want to put into the clipboard, and for that we use e.clipboardData.setData(type, data). In our case the type used is just plain text (test/plain) but you could use other rich data formats if you need.

You will also notice at the bottom of the code, we focus back on the canvas to shoot us back to where we were. This completes the cycle:

Event.observe(document, "copy", function(e) {
    var selectionText = _editor.getSelectionAsText();
 
    if (selectionText && selectionText != '') {
        e.preventDefault();
        e.clipboardData.setData('text/plain', selectionText);
    }
 
    $('canvas').focus();
});

The cut action is identical bar the step that gets the data into the clipboard. In the case of cut we also add a step to delete the current selection:

_editor.ui.actions.deleteSelection(selectionObject);

Paste is also quite similar, but instead of adding to the clipboard we are getting the data out, and for that we use:

e.clipboardData.getData('text/plain');

Now we are good to go. You can see the code in its entirety here.

You may notice some of the wiring up of the two styles (Firefox/WebKit).

We cheat and do a simple test:

setup: function() {
    if (Prototype.Browser.WebKit) {
        this.install(new Bespin.Clipboard.DOMEvents());
    } else {
        this.install(new Bespin.Clipboard.Default());
    }
}

We favour feature detection over browser detection of course, but here it is a little painful as the real test is to to see if the event in a copy/cut/paste/ has a clipboardData object associated with it. That’s a little too deep into the onion layers, so we punt.

We would love to use this standards approach with Firefox too, and we almost can. MDC has docs on support for this, such as paste but the rub is in this bug that states how onpaste doesn’t let you getData() back, which makes it useless for all bar signaling when the user wants o paste something. PPK does a good job talking abou the various quirks with these events in general.

We still have some problems though. We handle the copy/cut/paste menus and keyboard shortcuts perfectly, but our UI has our own icons that try to kick off these actions. This is where we need to get zeroclipboard going, and then we are done (in WebKit).

Phew. That turned out to be a long explanation. Fun times on the Open Web! What am I missing? What tricks have you run into? Inquiring minds want to know!

More articles from Bespin

We are learning a lot from Bespin and beyond. Here are posts from the team and community that you may find interesting:

26 Responses to “Supporting the system clipboard in your Web applications; What a pain!”

  1. Bill Mill Says:

    So, in summary, can you or can’t you make it work with firefox in any sort of reasonable way? I’m confused.

  2. Boris Says:

    Fundamentally, the problem you’re running into is that being able to place text controlled by a web page into the system clipboard without the user selecting it and copying it is considered a security bug in Firefox.

    So you need to make sure that whatever setup you use doesn’t involve doing that, or anything that appears to be that to Firefox.

    This is a hard problem.

  3. Boris Says:

    To be precise the hard problem is how to design an API that would work for you, and couldn’t be abused by a malicious web page. I’d love to see any proposals you have in m.d.platform.

  4. Jeff Schiller Says:

    Missing the link for W3C DOM Level 3 Events spec…

  5. Jeff Schiller Says:

    … and actually I couldn’t find ‘beforepaste’ described in any W3C specification

  6. Emmanuel Pirsch Says:

    Did you explore a signed Java applet (jnlp style)?

    This may not be a popular choice, but by signing the applet and specifying which permissions you seek. The user will have the choice to grant permissions permanently or not.

    This will also give you access to binary (images for example) copy and paste.

    Note that you really should use the applet tag with a JNLP file as this will give you a much better applet environment (Java plugin) than the browser applet environment.

    From there, you can easily communicate from/to javscript to/from the applet. And this will work with any A grade browser.

  7. Jeff Schiller Says:

    By the way, I applaud this noble experiment, but…

    I guess I haven’t heard a solid reason why canvas was used over more conventional HTML 4.01. Even over here where you justify it by saying: We need control. Control over what exactly? Font? Text Color? Background Color? Hover Tooltip? What else?

    It seems like Google Docs does just fine with giving me an editable text document that is cross-browser, text which can be richly styled, text which the browser and system know is selectable, copyable, linkable, can be checked in a spell-checker, can be ubiq’ed, etc. And it’s a document that can be edited by multiple people simultaneously. It seems they do it using nothing but HTML 4.01 elements.

    It seems like a lot to give up – and I’m not sure all the things you’re trying to gain.

    You mention performance… I didn’t do the experiments but I have a hard time believing that a page-sized div of HTML elements (spans, etc) that is dynamically changing as you scroll wouldn’t be as performant as programmatically formatting, styling and drawing the text on a canvas.

  8. Ben Galbraith Says:

    @Jeff Schiller: Do the experiments, read the blogs and comments from folks who have tried to do it (Daniel Glazman comes to mind immediately, others may be out there), and I think you’ll come around.

    You might also note that this isn’t a novel approach; Eclipse, IntelliJ IDEA, NetBeans, Visual Studio, and many others have all hand-rolled their code editors from scratch.

  9. smaug Says:

    Drag events and clipboard handling aren’t specified in W3C DOM Events spec.
    HTML 5 draft does include them.

  10. smaug Says:

    Oops, didn’t mean drag events but copy+paste, those are (also) in HTML 5

  11. smaug Says:

    And I misremember that :(. copy+paste events are from Microsoft.

  12. Robby Colvin Says:

    A little off topic, but do you know where I can buy those onesies?

  13. Wladimir Palant Says:

    Yes, getting clipboard access from a web application is a security hole – and the fact that you can still do it with Flash is troubling. If I were you I wouldn’t count too much on that hack, Flash 11 will likely break it again.

    However, why can’t you use existing mechanisms? Since you draw everything in the canvas yourself, I don’t think you care too much about input focus. Why not have an invisible text field that takes focus instead of the canvas? Whenever the user selects something in the canvas you copy the selected text into the text field and select it there. This way when the user presses Ctrl+C it will work automatically. Cut and paste operations in the text field can be detected and you can easily get the pasted data out. I realize that this isn’t quite as simple as I made it sound but at least that would be a stable approach.

    Copy and paste buttons on the toolbar are a different thing of course, for that you will have to work around existing security mechanisms – meaning that it might be better to consider some different UI approach.

  14. Dao Says:

    What Wladimir said. Given that you want the hidden textarea anyway for accessibility, that seems like the way to go.

  15. Arthur Blake Says:

    Where can I get those copy & paste onesies?

  16. Tom Robinson Says:

    In Cappuccino we bypass the oncopy family of events completely, and simply capture the key combinations, then focus / select the hidden input field and fill it with data appropriately.

    This works in every browser, but there are a few downsides, namely copy/paste can only be triggered by the key commands, not the “Edit” menu items.

    Perhaps a combination of of the two techniques would work better, using the oncopy/paste events if they’re supported, falling back to key events.

  17. voracity Says:

    I find myself constantly oscillating between thinking that this is a great and innovative project to being worried that the choice to snub *hackable* web technologies (with what appears to everyone like Mozilla’s blessing) will do more harm to the web than Microsoft’s neglect of IE6.

    When I inspect BeSpin with Firebug, I see a div element with id=editor that contains a canvas element with id=canvas — and that’s it. I can’t use Firebug to manipulate the editor (like resize, move or delete elements). I can’t put CSS declarations in my user agent style sheet that will have any effect on the BeSpin text editing — and furthermore, I can’t use any extension that manipulates fonts, colors, etc. on my behalf. Even Firefox’s Zoom Page function has no effect on BeSpin. When I run a “Find on Page”, I can’t find any of what I’ve typed. Even the copy & paste problem you describe in this post arises partly because you’ve spun your own UI. What about testrunners? I use Selenium IDE to automate my website testing. Completely useless with BeSpin. The ScrapBook extension, which lets you save webpage clippings? Useless again with BeSpin. Spellcheckers? Useless. Context-sensitive searches? Useless. The list of browser features and extensions that this breaks just goes on and on.

    And then there’s Ubiquity. Oh what irony that Ubiquity becomes completely useless on a fellow labs project given what the word ubiquity means!

    BeSpin is still hackable in one very important regard: because it uses the browser’s Javascript engine, I can use something like GreaseMonkey to alter it’s behaviour. But BeSpin is its own system — I have to learn the BeSpin system in order to do any hacking on it.

    What happens if everyone starts thinking this kind of thing is a good idea and developing their own BeSpin-like projects? We’ll have 10000 different websites with 10000 different systems that people have to learn in each new case to do something basic like change their font!

    To be fair, you’ve proposed adding a custom DOM within the canvas. This may address some of my concerns, but unless you are proposing to make that custom DOM a wider standard, it really doesn’t address anything. You may as well just dump out plain text, because the tags are meaningless by virtue of having no common interpretation. (Because that’s what meaning is!)

    I don’t wish to sound too critical. I realise this is an experiment and I think it’s a great exploration of what’s possible and is a very promising project in itself. But unless you address hackability, then I cannot support the project.

    (Aside: I find myself constantly trying to defend web hackability. Whether it be because of Flash, Silverlight or Java, or information hiding and non-mutability in ES4, or security practices that haven’t been subjected to a proper cost/benefit analysis. Where is the FSF when I need them? :) )

  18. voracity Says:

    Apologies, Firefox’s zoom does work, it just looks horrible.

  19. Dave Johnson Says:

    We have done something similar Dion where we capture the ctrl + c keydown event on whatever DOM element you want then put whatever text you want to copy into a hidden textarea and focus the textarea with the text selected. The keyup event will cause a native copy to happen and the selected text will then be on the system clipboard. Paste works in a similar way.

  20. Chris Thornton Says:

    For those interested, here is my article summarizing the Flash clipboard vulnerability. http://www.clipboardextender.com/defective-apps/clipboard-virus-not-exactly-but-still-dangerous

  21. Barney Carroll Says:

    Obviously your app has its own special wares to bring to the table (or you wouldn’t be interested in developing this), but the way I see it there’s two ways to go about this:

    1) Access to the global OS clipboard proper – I’d be very worried for the future of the web if you succeeded in doing this while sticking to non-proprietary W3 tech.

    2) Create your own clipboard + cut & paste methods that stick to your app – context menus & events bound to Ctrl+C/X/V operating with mouse coords and focus respectively on a textarea.

    Regards,
    Barney

  22. Jesse Ruderman Says:

    “contentEdible” is a delicious typo.

  23. Sam Foster Says:

    @Voracity – good points. I agree that a part of the “open” in the Open Web is being lost here. Being able to poke, prod and tweak a thing in the browser, at runtime is very much a part of what gives the Open Web its value. No black boxes.

    But that’s a digression from the subject of this post. The applet seems to be the way to go, if you cant do it all with a hidden field. The applet and js api you wrap around it just need to bridge the gap – put a facade up that smooths away the funkiness you describe here, until such time as sanity is brought to the subject. That would give you your IE compatability too (when you need it).

    I know everyone has this knee-jerk reaction when it comes to java applets, but seriously, why not?

  24. Pies Says:

    Just loose the UI icons and use the hidden input/textarea solution. I don’t see a point in programmer’s editor having icons for commonly used functionalities like Copy/Paste.

  25. John Dowdell Says:

    From what I understand, if the browsers tracked user-initiated JavaScript events, and could pass the info to plugins, the old way would still work.

    (Or you could just put the button in Flash, which can track user-initiated events.)

    That “zeroclipboard” hack… that’s a clickjacking exploit, isn’t it? Browsers really need to guarantee that What You See Is What You Click. Hacking atop the fault doesn’t seem sustainable.

    jd/adobe

  26. brett Says:

    any idea if mobile safari supports clipboard access? it’s not obvious form the docs if they do…

Leave a Reply

Spam is a pain, I am sorry to have to do this to you, but can you answer the question below?

Q: What is the number before 3? (just put in the digit)