Sep 28

Chrome Speak To Site; Give any input the power to listen to you

JavaScript, Open Source, Tech, Web Browsing with tags: , , 3 Comments »

Paul Irish gave a fantastic updated State of HTML5 talk at JSConf.EU. It is packed full of demos, including sharks with freaking lazer beams!

At one point he showed off the WebKit support for <input speech> implementation that allows you to talk into an input area. You click on the microphone, speak in, and it will get translated for you with the results. I am not sure if you can tweak how the translation is done (choose a Nuance vs. Google vs. …. solution for example), but it definitely works well out of the box.

speak-to-site

I was surprised to see this already landed in my developer-channel Chrome, so I was incented to do something with it on the plane trip back from Berlin to New York City. Something simple would be to give the user the ability to enable speech on any input. I whipped up a Chrome extension using the context menu API, but was quickly surprised to see that there isn’t support in the API to get the DOM node that you are working on. Huh. Kinda crazy in fact.

Then the whizzkid antimatter came to the rescue with his cheeky little hack around the system. Here is how it plays out in the world of this extension:

The background page

First we enable the context menu on any “editable” element (vs. anywhere on the page, on any text, etc), and when clicked we fire off an event to the content script in the given tab:

<script>
chrome.contextMenus.create({
    title: "Turn on speech input",
    contexts: ["editable"],
    onclick: function(info, tab) {
        chrome.tabs.sendRequest(tab.id, 'letmespeak')
    }
});
</script>

Catching in a content script

A content script then does two things:

  • Listens for mousedown events to keep resetting the last element in focus
  • Catches the event, and turns on the speech attribute on the target DOM node
var last_target = null;
document.addEventListener('mousedown', function(event) {
    last_target = event.target;
}, true);
 
chrome.extension.onRequest.addListener(function(event) {
    last_target.setAttribute("speech", "on");
    last_target = null;
})

Wire-y wire-y

Of course, it all gets wired up in the manifest:

{
    "name": "Turn on Speech Input",
    "description": "Turns on the speech attribute, allows you to speak into an input",
    "version": "0.1",
    "permissions": ["contextMenus"],
    "minimum_chrome_version": "6",
    "background_page": "background.html",
    "content_scripts": [{
        "matches": ["<all_urls>"],
        "js": ["input-speech.js"]
    }]
}

This trivial extension is of course on GitHub (I want git-achivements after all! :).

A couple of things trouble me though:

  • The microphone icon should sit on the right of the input, however when dynamically tweaked like this it shows up on the left by mistake [BUG]
  • I have also played with extensions such as Google Scribe. Adding icons like this doesn’t scale. Having them show up all the time gets in my way. I think I want one ability to popup special powers like scribe completion, or speech-to-text, without it getting in my way
  • When services are built into standard elements like this, it feels like I want to have the ability to tweak how they work (with great defaults of course, as 99.9999% of the time they won’t be changed.

You?

Sep 26

Promote JS; Give newbies a chance :)

JavaScript, Tech No Comments »

javascript on search

Do a search for JavaScript and you find a painful set of returns. The worst offender is having Java results show up. Ouch. (Remember: Java is to JavaScript as Ham is to Hamster!).

Compare to a search for Java, or C#, or Ruby, (or ….).

Ouch. We need better. To begin with the pirates of JSConf (lead by the awesome Mr. Chris Williams!) have started a grass roots Operation SEO at Promote JS. Go there, and help the cause by doing embeds like the following:

JavaScript JS Documentation: JS String replace, JavaScript String replace, JS String .replace, JavaScript String .replace

If we can do half of the work to promote this as we have with the awesome VaporJS library, then we should be in a better place.

Step 2: Come up with a fantastic landing area for JavaScript.

Sep 25

Microsoft, Please work with us on pinned sites like this

Microsoft, Tech, Web Browsing 5 Comments »

There has been a bit of chatter regarding how IE9 implements pinned sites, namely that this isn’t the way to go:

<meta name="application-name" content="Ars Technica"/>
<meta name="msapplication-starturl" content="http://arstechnica.com/"/>
<meta name="msapplication-tooltip" content="Ars Technica: Serving the technologist for 1.2 decades"/>
<meta name="msapplication-task" content="name=News;action-uri=http://arstechnica.com/;icon-uri=http://arstechnica.com/favicon.ico"/>
<meta name="msapplication-task" content="name=Features;action-uri=http://arstechnica.com/features/;icon-uri=http://static.arstechnica.net/ie-jump-menu/jump-features.ico"/>
<meta name="msapplication-task" content="name=OpenForum;action-uri=http://arstechnica.com/civis/;icon-uri=http://static.arstechnica.net/ie-jump-menu/jump-forum.ico"/>
<meta name="msapplication-task" content="name=One Microsoft Way;action-uri=http://arstechnica.com/microsoft/;icon-uri=http://static.arstechnica.net/ie-jump-menu/jump-omw.ico"/>
<meta name="msapplication-task" content="name=Subscribe;action-uri=http://arstechnica.com/subscriptions/;icon-uri=http://static.arstechnica.net/ie-jump-menu/jump-subscribe.ico"/>

Some are calling for using the menu tag, others for
a link tag that points to an external resource that defines the menus.

There are pros and cons to the different approaches (reusing menus across the site instead of embedding inline, [or just using <a> as Sam Pullara mentions!]). For me, the details on the implementation aren’t what is important here.

What matters to me is how Microsoft goes about adding these features. It is their browser, and they can do whatever they want. However, if they really want to work with developers, they could have worked with us (and other browser vendors) on the feature. I am all for trying things out and not standardizing anything prematurely, and I am frustrated when browser vendors don’t experiment more sometimes, but man.

This is a “beta” of IE9, so I hope that there is time to work on this together. If other browsers are interested in doing something similar in the future, it would be nice to abstract the task at hand, which is denoting what actions are available in the given page (or on the given site as a whole). If the msapplication prefix is used in a “before we work this out” way, that is probably OK, but I really don’t want to go down the path of multiple ways of defining this. I personally don’t enjoy seeing -webkit-*, -moz-*, -o-* in CSS. So much repetition that never goes away. I would much rather have someone grab foo and let people -vendor-foo if they disagree until things work out.

Microsoft, if you have worked with other browser vendors and developers on this, I forgive you; but if you haven’t and aren’t willing to listen to the community, I fear for the history repeating itself.

Am I off base?

Sep 17

Crowd sourcing security and permissions; IE9’s baby steps

Security, Tech 7 Comments »

The Web lives in a sandbox. An important one. It means that you can click around on links and “feel” safe (that is, until you go to a talk by Jeremiah Grossman).

On your desktop however, you have a binary situation. You either run some code that can do pretty much anything (ignoring file permissions and virtualization etc) or you don’t run that code.

One of the dialogs that I dislike is from OS X when you “download something from the Internet”:

internet permission

I would love to see the stats that Apple may have on how people react to this. I can imagine a scenario where something has mistakenly been downloaded and run, but I would guess that 99% of the time a user goes through the following:

  • Click on something to download an app
  • Run the app, because I wanted to, hence me clicking on it
  • I have no information here…. so I think I will just click yes!

Some will yell, “think of the children!” and that if this popup has stopped ONE virus or malware through, it is worth the pain for the other millions of up pops.

Surely we can do better. Of all of the features in the IE9 beta, one that may not shout out as you as much as fancy fast chess or goldfish bowls, is how they manage this situation.

In Dean’s whirlwind keynote presentation (very nicely produced!) he showed a feature where clicking on a particular .exe wouldn’t alert the user based on crowd data. Basically, if enough people have used foo.exe and it is trusted, it can Just Get Through.

The virus scanning world is doing a lot of this on the backend. Places with scale can monitor the crowd and do a lot more than we have done now, and I can’t wait to see what comes of this.

We have talked about social permissions before. It will be great when I can see that Jeremiah Grossman is using a particular application for example ;) I look forward to a way out of app permissions hell.

Sep 16

#newtwitter, IE9, and the change in user experience expectations

Tech, Web Browsing 2 Comments »

Yesterday’s announcement of what was immediately dubbed #newtwitter has crept up on me in its significance.

While Twitter has been growing in mainstream significance and popularity, it hasn’t managed to adopt a strategy that clearly aims the company towards mass market success. I think #newtwitter changes that, turning the site into a rich information discovery platform, if you’ll excuse the buzzword bingo. The new design is a pleasure to use, and encourages a kind of deep exploration of the data within Twitter that has previously only been exposed in bits and pieces by third-party applications. Browsing Twitter is now as rewarding as communicating with it.

This comes from Alex Payne, former Twitter API architect and all around top notch bloke. The new Twitter interface came at an interesting time for me. You see, Ben and I are working on upcoming presentations for Web Directions, JSConf.EU, NY JS, and Web 2.0 Expo. We have been spending some time analyzing the interesting worlds of apps vs. mobile Web, and how the worlds are converging in interesting ways. After the Twitter for iPad came out:

it was a stark contrast with Twitter.com:

twitterdotcom

On the iPad, the team had done a great job thinking about the feel of the application and how people sit back with their tablet devices and interact with them via touch. The stacking system is fun to use, and although some times it feels a little rough, it feels that way in the way that things in the physical world can feel rough. I like it. Of all the features, the only feature that truly matters to me is the inline preview of URLs. As soon as I experienced it, I knew it would change how I use Twitter. Flipboard is great, but is a fun sit back experience that I actually tend not to use. Twitter for iPad however gives me a more practical experience that goes beyond 140 characters. The Web joins me in the application rather than shooting me out to other tabs in the browser, or apps in the silo space.

After using the iPad version I started to ponder how different it was both from twitter.com and desktop clients (Tweetie for Mac, echofon, etc). I am not a TweetDeck “let me view lots of lists!” type of user. My desktop client sits in the corner, an appendix. I like how echofon uses draws to extend the space, but I now want to be able to change the size of the window and when large enough, flip to a tablet view. There is no reason that the desktop application couldn’t morph nicely and let me use the same application for my different use cases (small area of the screen to view tweets going by, extended version to immerse in the experience). When looking at twitter.com, the boring list of Tweets seemed poor compared to FriendFeed and Facebook’s expansion of that area. Those sites did smart things based on the type of content in the status update, they show comments right there, etc. Twitter has been notoriously awful for tracking the conversations being had in @replies. Changing this dynamic will change the conversations.

Just as I was looking at the touch iPad experience side by side with twitter.com and wanted to shout “the Web can do that too!” Twitter came along and implemented a fantastic rewrite of the site. The new Twitter brings an experience into the site that makes it so much more usable than 140 characters with a link.

Steve Souders just posted about his movement from blogging to Twitter. Twitter has very much changed the way I view blogging too. On Ajaxian.com I wanted to bring the community together to discuss the Ajax revolution. One of my goals was “if a developer only has a few minutes to see ‘what is important today?’” they could come to Ajaxian and feel like they are keeping up. There were other goals too (e.g. run The Ajax Experience as an excuse to get the top Ajax talent together to further the Web). The site had some exclusive content, but it was very much an aggregator, a curator. In that vein, my twitter feed started to become just as good as the website for the goal of people keeping up to date. With the new Twitter interface, that is even more true. That isn’t to say there isn’t room for more however. We have had visions of doing much more on the social community side, and hopefully one day we will get a chance to implement those visions. Whatever the case though, Twitter will be a real part of the solution.

ie9

Our users are expecting fantastic experiences. Once expectations are changed, the previous world is unusable. The day you played with Google Maps was the day you couldn’t deal with MapQuest. It felt archaic in comparison, but a day before hand you were fine with it!

The apps revolution has pushed fantastic experiences on us. With webOS we are showing that the Web can be a fantastic native runtime for these experiences. Today, we got to see Microsoft get behind the Web in a huge way.

“With HTML5, you can make as rich an experience as an app.”

At one point, Microsoft came along and pushed the Web forward with IE4, 5 and even poor old 6. However, once they had the market-share they gutted the team and moved on to WPF and Silverlight. The Web was the ghetto but to do beautiful, rich experiences…. use Silverlight and WPF! This is why it is amazing to see them pimp the beauty of the Web. You kinda have to rub your eyes in this turn around! Next, I hope that the Windows Phone 7 folks get the memo ;)

Now all of the top browsers have fantastic runtimes that are competing heavily. Fantastic news for us all. The Web is going to change in a huge way, and as developers we need to step up to the opportunity to deliver these experiences across various form factors. Right on.

Sep 11

CommonJS: Bridging client and server, and why JavaScript desperately needs first class modules

JavaScript, Tech 4 Comments »

millenium bridge

(function(require, exports) {
  "use strict";
  // ...
  exports.x = 21;
}).apply({}, typeof require === 'function' ? [require, exports] :
               [function(g){return function(id){return g[id];};}(this._modules = this._modules || {}),
                this._modules['example/foo'] = {}]);

Ouch. This is one of many variants of boilerplate hacks that folks are using to get CommonJS modules playing nicely with browsers. The CommonJS list has been sparked by the question of client vs. server, and how to deal with the two worlds.

First, “Mark Miller has posted a recommendation on the TC39 wiki for CommonJS module boilerplate that permits modules to be crafted for use as <script>”.

And, then we have Kris Zyp’s proposal for an asynchronous module definition “based on the ideas from Transport/C and recent changes discussed.”

There seem to by two high order bits:

What is the focus of CommonJS?

The project started as “ServerJS”, and many server side JS folks started discussions. However, then other browser JS lib folk got into the mix. Folks who deeply care about module systems for the client as well as the server.

There are varying degrees of interop here. On the one hand folks want to be able to have the work that “makes Foo a module” work in both contexts. But further down we run into issues like: how does legacy code work? Do you have to wrap it? How? And finally, the browser needs to be able to load code asynchronously whereas the server doesn’t have the same issue.

Node is taking off like wild fire. Should Ryan be stopping to think about the browser? Does he really care? Or is he more focused on making a great event driven server environment that uses JavaScript?

What common libraries should we be creating as part of the “SDK” of JavaScript so to speak?

How do we hack on support :/

When we get into the “how” it gets messy quick. Messy can be hand-waved away if you assume that everyone will be using build tools and the like, but a ground swell of comments have been about the developer ergonomics of working on projects. How is it to debug and write applications? Do you get the basics: correct line numbers, no need to run builds all the time, etc? And then there is the performance aspect of the final production code.

All of this has just shown me how desperately we need a first class concept of a module beyond the client-esque “script” and function wrappers. Could you imagine showing the apply() code from above to a Java programmer? “Erm, you can’t just import?”

ES4 tried to give us programming units, packages, and namespaces. How about ES.next giving us one simple system to get us out of this CommonJS mess?

Sep 07

How is your Time Porfolio? Staying Healthy and Sane in Tech

Personal, Tech 2 Comments »

time spiral

I really enjoyed reading Alex discuss staying healthy at a startup and his quest for balance.

I am going through a similar journey myself, and it isn’t easy! I seem to be going through a cycle (often recursively) of:

  • Measure
  • Plan
  • Execute.

I don’t have a lot to offer the space. There are a ton of books on the subject, and I am far from a master, but here are thoughts that I have been having as I watch my own progress:

Measure

I have a tendency to jump into things and want to fix everything at once. In this case, I normally sit down and write out a new process and charge off on that path. This time around, I tried to start with measurement. What is the realistic starting point that you find yourself in? To keep it simple, I would track where I am spending time (you can do so in your calendar, in notes, whatever). It is hard to hold off judgement as you see the reality of your time portfolio start to time through. The act of tracking time will quickly have you a little shocked at where your time goes. Draw out an ideal time portfolio before hand, and see how it compares (not just for work, but everything in your life. You are only one person :).

A friend was talking to me about J.D. Rockfeller and how he measured his businesses. He discussed how most businesses don’t measure anything. Some measure where they are. Better ones measure trajectory, Great ones measure their acceleration.

I now try to have that in the back of my head when I think about measuring goals. For example, the dollars that you have in the bank is interesting, but to plan, how much better is it to know how it has been growing, and how FAST it has been growing.

Plan

I really liked this segment of Alex’s post:

I’ve found that time management has little to do with “lifehacks” and how you manage your email inbox and more to do with prioritization, saying “no” to people, and clearly communicating the expectations you have for yourself and others.

Yes! It isn’t about gimmicks.

I find that a lot of people have a quick look at “GTD” and think all they need to do is write down tasks that need to be done. The core principles go beyond that and discuss much more. I have fallen into the “what am I doing today”-only trap. What happens when I am planning in the weeds only is that I keep putting one foot in front of the other quickly, but if you don’t look up, you find yourself in the wrong place when you eventually do!

The plans that I am trying to make now have different levels to them. I have the high level goals, and then closer objectives with tasks falling out that can get me baby steps to my goals. Family Wealth is a book that has you think about family planning in a very different way, and time span. Think: 100 years, not 20 years. I have found that if I can articulate and have long term goals well thought out, then when I come down the stack and do more near term plans, I can measure them against the big picture. There are other side effects too. I can be more tolerant to tasks or things that would bug me, as I know that the act of participation or completion are getting me closer to goals.

Execute

Habbit. I have a simple process that I keep tweaking (test a new process, measure! Don’t try to start with The Right Process). I have the process written down simply, and every week has me visit the process and go through the motions:

  • what are my big picture goals, what metrics to I have, how am I doing?
  • Now, what needs to happen this week? How can those be measured?
  • Finally, what needs to happen today. At this point I insert the short daily planning process.

This is my new kata. This gives me repeatability.

It has also shown me a lot. For example, we all know that too many meetings are a pain. When you start to measure, you realise that meetings can be Pure Evil! :)

Earlier, I tweeted this: “MEETINGS: A way for someone to get their priorities pushed on your TODO list.”

Suddenly all of those great plans for the day that you made, that are traceable to higher level goals, are thrown out of the window. This is where you have to get brutal and say “no”. You have to block out time on your calendar to map to your goals. Whatever it takes. If you have to give people time allocations that everyone can spend on a meeting, so be it. WHAT EVER IT TAKES.

Hard work

As a developer, I often look to tools to solve problems. Is there a way to create a new social platform to make this all easier? I do think that better tools in the arsenal could help, but these issues seem to come down to putting the time in and making a habbit. If you put the time in and want to make a difference, you can use post its or Notepad and be effective. If you don’t put the time in, you can use “The 7 habits of highly awesome people social game dynamic money making web 2.0 dot com” tool and fail.

May we all reach out highest goals. Are you going in the right direction?

Sep 02

Onerous terms

Apple, Facebook, Tech 4 Comments »

Steve Jobs on Ping and Facebook:

Facebook wanted “onerous terms that we could not agree to.”

but when I read this, in another time I was reading about other Onerous silly terms:

carmack onerous terms