Birth of a Platform
I think there’s a chance that the reasoning behind the now infamous change to Section 3.3.1 of the iPhone Developer Agreement is much deeper than the community has thus far considered. There’s a pattern that Apple has been following since the beginning of the iPhone project that may be hiding “the truth” in plain sight.
They’ve been slowly training us developers to stick with the documented stuff and use their higher level APIs. They want us to accept their abstractions and work within them. This is usually rationalized under the guise of safety, compatibility, and quality control. Those are fine and acceptable reasons by themselves, but what if there’s another purpose lurking behind the curtain?
I think there’s a chance that Apple is slowly building Objective-C into a managed environment similar to Java/.NET. At some point in the future they could define an Objective-C HD (or whatever :P) that no longer maintains total compatibility with C. Since they use LLVM a lot now, they can even use that to analyze your code to make sure that pointer accesses are safe and controlled. Anything that isn’t safely confined to your own app would be an error. Access to the Objective-C runtime functions could possibly even be revoked. After which point, Objective-C HD no longer compiles to machine code but instead to an intermediate representation.
By doing something like this, they can abstract the actual underlying CPU hardware and architecture out of the applications themselves as well as maintain a truly safe sandbox where private and undocumented APIs simply will not be allowed to work. Apps on the App Store would be submitted in this intermediate format which they can translate into the machine code that’s native to whatever CPU happens to be in the device you’re downloading the app for or they could simply put a JIT in iPhoneOS (although there’s no reason to waste the CPU cycles on the device if they can translate them once on the backend - at least for mobile stuff).
Ultimately OSX itself would probably go this way as well. Each new app would become sandboxed to public and documented APIs and the OS transforms from the traditional thin hardware abstraction layer to a much thicker platform abstraction layer which provides high level and clearly defined services. Of course due to legacy constraints, I would expect an OSX transition of this nature to take many years and be a more gradual process. The iPhone platform is a great way for them to experiment with the idea since it’s been closed and heavily regulated from the start. (I do not think any of the closed-ness of the platform has been an accident and nor do I think it was done for malicious intents of evil-like control that pundits attribute to Steve Jobs and Apple.)
(Ultimately, I think the entire web may transition from HTML/Javascript/CSS to a kind of standardized instruction format like a bytecode, but we’re probably a decade or more away from that if it should ever even happen.)
So how does Section 3.3.1 lead me to this? Well, if Apple was getting ready to close off and eliminate total access to the hardware itself, they certainly don’t want people writing tools that target the CPU directly, generating machine code, or making other unsafe assumptions about what your code can do just because it happens to be technically possible to do right now. It’d make it harder for them to make this transition and it’d also lead developers to potentially wasting a lot of time building something that won’t be useful at all in the (near?) future. The contractual restrictions on private and undocumented APIs were only the start. Section 3.3.1 may be a clue that the next step will make those unapproved APIs entirely inaccessible and impossible to actually use at all.
Apple wants to reinvent computing. You didn’t think they’d just constrain themselves to the end-user experience, did you?
May 21st, 2010 at 11:04 am
Just commented on this at my blog: http://brockerhoff.net/blog/2010/05/21/re-ipad-time-4/
May 21st, 2010 at 4:46 pm
Just look at http://cappuccino.org/ for the future of Cocoa (web).
It’s pretty obvious that one day XCode will be able to compile directly into HTML/Javascript. And they are taking steps to make that happen.
May 21st, 2010 at 4:47 pm
You didn’t mention MacRuby at all?? A lot of what you’re speculating is what MacRuby is being built to do - you might want to visit http://www.macruby.org/ for more details!
May 21st, 2010 at 4:55 pm
I think it’s more likely that by getting you to stick with approved methods, they can change HW more easily… and cross-pollinate apps with new devices. Did you see how easily apps ran on iPad? Imagine if all your apps now run on Mac OS X as the replacement for Dashboard widgets.
May 21st, 2010 at 4:59 pm
Interesting idea, but I can’t see this ever happening for pure C code and there’s no restrictions whatsoever on using pure C code as long as it doesn’t call undocumented APIs.
May 21st, 2010 at 5:02 pm
The whole point of a public API is so that a client isn’t dependent on implementation details, so aren’t you kind of stating the obvious? The only question is whether or not Apple is implementing a “managed” platform, and there’s no evidence of that. They’ve always encouraged third-party developers to avoid private APIs.
May 21st, 2010 at 5:02 pm
I think you’re on the right track, though I doubt if they’d completely cut off all “unsafe” C-level access; there’s always got to be some kind of escape to “bare C” where appropriate.
HTML5/CSS/JS are *already* a real “virtual machine” for the web. The machine language is Javascript, and the display model is HTML + SVG + Canvas + WebGL (take your pick). You can see it happening with GWT compiling Java to Javascript, and with things like Cappuccino compiling Objective-J to Javascript (though it’s a pretty thin layer), etc.
May 21st, 2010 at 5:17 pm
@Kevin: Apple could potentially redefine Objective-C and remove a lot of the unsafe bits from C and declare Objective-C it’s own beast and no longer a superset of C. I’m betting a lot of existing code would still work - especially if the rules were applied at a slightly higher level than just syntax. For instance, if they can prove with LLVM or other tools that your pointer operations in some functions are always confined to the app’s own memory space then it’s fine. Stuff like that. Just speculating, of course.
May 21st, 2010 at 5:22 pm
@Chris: Yeah, Javascript is sort of a bytecode of the web as it is. I think, though, there’s a chance that eventually a browser becomes just a container for Canvas. You could implement HTML/CSS on top of canvas and do it with libraries written in Javascript itself! If something like that happens, then the browsers don’t even need to implement HTML/CSS anymore. A WebKit-like open source/shared implementation written in Javascript could appear, be used by everyone, and sites could eventually include their own rendering engines (or none at all) depending what they wanted to do. Maybe something like that will happen. Who knows?
May 21st, 2010 at 6:06 pm
Geez, I (kind of) hope not. One of the things I like most about ObjC is that I feel that it doesn’t make me choose between having a nice language and getting the performance of lean C code. (If ObjC had namespaces it would almost be my favorite language, if it also had numpy/matlab array slicing syntax it would definitely be my favorite). How many runtimes environments approach the speed of C? I do a lot of work in python, but boy, can some things be slow. .Net is adequately fast, except for the stuff requiring the best performance. I’ve read good things about LuaJIT.
Hopefully the static analysis of Clang will just get good enough that it’ll catch most programming mistakes without forcing code to run in a managed environment.
May 21st, 2010 at 6:13 pm
Yeah, well, maybe, but as others have said I don’t think we’ll be losing C access any time soon, and whilst we have C what would be the point in fragmenting the APIs into separate Objective-C and C portions?
Also bear in mind the new threading support and dispatch being included in iPhone OS 4. Apple likes speed and C is pretty unbeatable for that.
May 21st, 2010 at 6:13 pm
MacRuby is being built right on top of LLVM.
MacRuby accesses the Objective-C runtime directly - no bridge.
MacRuby can call Objective-C code, so you can still write or call C stuff if you have to, and do the rest in Ruby.
MacRuby is a project being run by Apple. Yes, Apple themselves are working on MacRuby - that alone is a strong indicator to me that this is the direction they’re heading.
May 21st, 2010 at 6:32 pm
So you’re basically saying that Apple is trying to recreate Smalltalk.
May 21st, 2010 at 6:46 pm
I don’t think so. Using C/ObjC gives maximum performance on limited hardware. The only thing that cocoa touch lacks is garbage collection. Any code that runs in a sandbox or runtime sacrifices performance and may even encourage developers to use private APIs when the abstraction fails.
Open-source toolchains like GCC and LLVM make compilation to multiple platforms much easier than it used to be and C is pretty much as mature and optimized as it’s ever going to get (Objective-C is little more than C with some clever macros). Apple aren’t going to make their own ‘Flash’.
Java was created by Sun as a weapon to brake Mircosoft’s monopoly on app development and .Net was created to try and migrate Windows developers away from the clunky old Win32 API. They both have very serious drawbacks. Apple have been through multiple CPU and architecture transitions (68k, PPC, Intel) without the need for such sandbox APIs.
May 21st, 2010 at 7:05 pm
@David: Pretty much, yeah. Objective-C was more or less designed as a pragmatic Smalltalk. Hardware and compiler technology has advanced considerably since then. They can go back towards Smalltalk and sacrifice little if any performance.
May 21st, 2010 at 7:25 pm
@Danny77uk: A sandbox does not imply that the code must be slower. It also doesn’t mean you can just get around it’s limitations with private APIs, either. If you could, it wouldn’t be a very good sandbox!
Don’t fall into the trap of assuming performance is a good enough reason to avoid doing something like this. Remember that Apple originally wanted everyone to write web apps only - and that was back when the iPhone CPU was very slow!
May 21st, 2010 at 7:58 pm
quote:
“So you’re basically saying that Apple is trying to recreate Smalltalk.”
Smalltalk is the only missing part from the original Xerox PARC GUI development that isn’t implemented in the operating system.
(I do not think this is the reason, but maybe Apple got nostalgic.)
May 21st, 2010 at 8:27 pm
Intriguing.
Re: “…the entire web may transition from HTML/Javascript/CSS to a kind of standardized instruction format like a bytecode..”
Putting aside the discussion of whether or not that’s a desirable future for the entire web, when it comes to web-deployed applications that’s already happened in some cases. Adobe Flash already does exactly what you describe. Also, consider Silverlight, JavaFX & Google Native Client.
I’ve been wondering if/when Apple might join that party.
May 21st, 2010 at 10:27 pm
Can someone tell me what the advantage of this would be? I’m honestly not a very experienced programmer, but from I’ve done with Java, I much prefer languages like C++ and Objective C.
@Sean
“Don’t fall into the trap of assuming performance is a good enough reason to avoid doing something like this. Remember that Apple originally wanted everyone to write web apps only - and that was back when the iPhone CPU was very slow!”
Yeah, but web apps were never meant as a serious solution. Are you telling me what we have now is just another intermediate measure?
@John Yanarella
Sounds like a plan. Everyone loves Flash, JavaFX, and Silverlight, right?
May 22nd, 2010 at 1:22 am
“Are you telling me what we have now is just another intermediate measure?”
From a high enough perspective, everything is an intermediate step.
…from a certain point of view.
May 22nd, 2010 at 3:37 am
[…] Original source : http://blog.bigzaphod.org/2010/05/21/birth-of-a-pl… […]
May 22nd, 2010 at 3:47 am
I’ll tell you why this won’t happen…. a C compiler is available for ANY processor, creating an objc compiler from C is very straight forward (original objc was all macros), from there, cocoa is a compile away.
May 22nd, 2010 at 6:43 am
With the original Mac, Apple said the same thing — this is the official API, and things not guaranteed in the API are simply not guaranteed to remain true with the next computer or the next OS. But it was a wild and woolly time then, and people wrote to the machine they had in front of them instead of writing to the spec. And then Apple introduced another model, or upgraded the OS, and things broke, and people screamed holy hell and blamed Apple.
This kept on happening, over and over again. Microsoft’s apps broke badly when the OS went from using 24 bits in 32-bit pointers to using all 32 bits, because Microsoft, knowing that there would never be 32-bit pointers, used the other 8 bits for their own purposes. Many apps broke badly when System 7 rolled around, because System 7 had large chunks rewritten from the ground up according to the spec. And the same thing happened with the PowerPC transition, and the Intel transition.
In short, Apple have learned the hard way that developers will prefer “it works, ship it!” over “it complies with the spec, ship it!” in 99 out of 100 development shops. They have also learned the hard way that when Apple changes things that it never committed to in the first place (such as undocumented APIs) that they get blamed even though it’s the developers’ fault for doing something that they should have done. So Apple’s stepping up and enforcing this.
May 22nd, 2010 at 9:19 am
[…] Seeking tο Understand » Blog Archive » Birth οf a Platform […]
May 22nd, 2010 at 9:52 am
As the focus of personal computing becomes ever more personal, it will increasingly receive the unwanted attention from malware authors.
A significant degree of happenstance and luck has kept the Mac malware free but inevitably it will fall.
I believe a major goal in the design and evolution of the iPhone OS is to not be the smartphone platform that needs antivirus, this is way way more important than winning the marketshare race, or making devs happy.
May 22nd, 2010 at 10:11 am
@Rain
Each of those technologies certainly do have a set of vocal detractors with valid complaints. And, sure, it is undeniably fashionable within the Mac community to scoff at Flash right now. However, neither the transient implementation flaws nor the disgust that web standards advocates might have for those proprietary platforms discount the fact that web-deployed managed bytecode applications are already a part of the web ecosystem. We can only hope they improve or replaced with better solutions.
Re: Sean’s notion of the browser as a container for Canvas with an HTML/CSS layout engine implemented and deployed as a JavaScript library is an interesting scenario where an web standards architecture might achieve what these other technologies have already attempted to varying degrees of success. It would be wonderful to see the progress of the web platform as a whole no longer hindered by the lowest common denominator browser.
To bring it back to Sean’s post, specifically to the idea of the evolution of Apple’s application deployment model - my original point was simply that it wouldn’t be the first time Apple watched others flounder at delivering quality and usability, and come along with a more focused and elegant solution.
May 22nd, 2010 at 5:54 pm
[…] Seeking to Understand » Blog Archive » Birth of a Platform […]
May 23rd, 2010 at 9:06 am
[…] Seeking to Understand » Blog Archive » Birth of a Platform […]
May 23rd, 2010 at 2:22 pm
Seeking to Understand » Blog Archive » Birth of a Platform…
Seeking to Understand » Blog Archive » Birth of a Platform…
May 23rd, 2010 at 5:02 pm
Forcing developers to use documented APIs as policy, and the trajectory of the C/Obj-C environment are two separate things, but there is _some_ merit to the claim that Apple would want to push development to a managed environment of some sort.
However be very clear that the managed environment has nothing to do whether the language is C, C++ or Obj-C, that’s the magic of LLVM. What it means is that potentially, the compiler can generate intermediate bytecode which is processor independent as is now the case with Java / CLR and thus make distributing binaries to different targets easier than it is now… not that it’s all that difficult now, but it could get messier as time goes on. The analysis dimension means that Apple could use metrics from it’s static analysis suite to refuse dodgy code, and that’s probably a good thing, and I might add totally in the hands of the developers since they have the same tools.
There will never be a day when Apple say to developers they can’t develop in C. This is just silly - The Obj-C runtime is incredibly flexible and cool to develop with, but it’s just crazy slow for things that require tight optimization (i.e. any tight game code, DSP, image processing etc - there’s a good reason why most of the stuff on Mac/iPhone that needs to go fast is done in C or C++. The message passing interface generates a page full of assembly, whereas a c function call is just two instructions (if not one!!), and a C++ vtable is marginally slower. C (and occasionally C++) is the language for system internals and backend engines, Obj-C is the language for applications and frontends, both play to their strengths in those areas.
But that doesn’t mean there won’t be a day when C isn’t generated into an intermediate bytecode… technically, that day could be today. The question for Apple engineers is whether the tradeoff for developers who require bare metal access to the cpu for performance reasons will get the same performance from a JIT or target hosted compiler… it’s too early to say yet whether that will be acceptable to those engineers.
Forcing developers to use documented APIs really has nothing to do with this, that’s just common sense from a quality control perspective. As John Gruber keeps pointing out, if you want an API public, file a radar bug for it, apple listens to that kind of stuff, and you’d be amazed how much code gets opened up as part of the normal development cycle… just look at the camera APIs in 4.0
May 26th, 2010 at 3:40 am
I think you can infer this by looking at the way OS X / Cocoa has developed over the years (think about the way that OpenCL, GCD and blocks can be combined to distribute work over potentially different CPU/GPU architectures), and as other commenters have suggested, by looking at MacRuby and Clang.
I also have a feeling that MacRuby is being lined up as the official 2nd language for Cocoa development (compared to other choices like PyObjC) - the particular point being that unlike other Rubies it can be compiled down to native code. (Etoile’s Smalltalk is interesting too, as that also uses LLVM and the GnuStep Obj-C back end, to make Smalltalk a seamless choice for Obj-C developers).
Ultimately, I can see a point where we should be able to write individual methods using the language of our choice, provided they support a compatible object model (which Obj-C, Smalltalk, Ruby and Python do).
But this is unlikely to be soon, and resource constraints suggest it would be much longer before it arrived on mobile (even Android is having to give native access for games).
Games will also remain a sticking point - most of them are mostly C++, not Obj-C - and games developers will keep them that way. C++ and OpenGL are relatively portable.
> It would be wonderful to see the progress of the web platform as a whole no longer
>hindered by the lowest common denominator browser
But surely it will remain so. IF HTML 6, say, defines a new JavaScript API for sound, similar to CoreAudio - you’re not going to use it until it’s commonly implemented (or there is a Flash-enabled shim).
It will certainly be a great day when we can depend on the lowest common denominator to actually implement standards in a consistent way - SVG in particular.
Then again, I’m not wholly convinced by ‘the web as application platform’ anyway. The App Store tells us that consumers still buy software, and watching non-techy friends with their iPhones I am reminded that it’s data that is the most important part of the web - URLs and HTTP are the most important layer, on which we can build many presentation layers.
Which is why it’s interesting to see people complaining that the App Store is about creating a proprietary web - I think the people who have that view think that the most interesting thing about the web is that it’s created a ’standard client’, not the web of information.
Cappuccino is very clever, but using Cappuccino apps I am reminded of the huge disadvantage of the web - 280 Slides takes an age to load - even after resources are cached locally - and uses up far more resources than Keynote to do the same work.
I can’t see any rational reason why anyone would choose to use Google Docs slideshow software over Powerpoint or Keynote, for instance, other than collaboration.
>Ultimately, I think the entire web may transition from HTML/Javascript/CSS to a kind of standardized >instruction format like a bytecode
There is talk of standardising an intermediate bytecode for JavaScript VMs - although this may be difficult as the different VMs take different approaches to optimising JavaScript - and older versions of IE don’t even use a VM.
I’d also like to see something that would allow us to use different programming languages, other than JavaScript, and a VM bytecode should make this feasible - although it may require a more flexible underlying VM to support other language features.
However, I don’t see the W3C backing anything that would damage the ‘web model’ of strictly separating data (XML), content (HMTL), style (CSS) and behaviour (JavaScript).
TBL’s bigger concern is with the bottom layers (semantic data tagging) rather than application development, which is one reason why the WHATWG went off on it’s own. I think he’s actually right on that one - the latter is an attempt to standardise application programming itself, while ignoring some of the important lessons we’ve learnt over the last few decades.
Another way of categorising it - it’s a bit like having the Cocoa APIs without AppKit to give a fundamental definition of what an application is.
May 27th, 2010 at 4:25 am
Seeking to Understand » Blog Archive » Birth of a Platform…
Seeking to Understand » Blog Archive » Birth of a Platform…