On his continuing quest for productivity and performance in the Java language, Brian Goetz, Java language architect at Oracle, along with Alex Buckley, specification lead for the Java language and Java Virtual Machine at Oracle, proposed a set of hyphenated keywords for Java. This stated goal is to provide a path to evolve a mature language in which adding new features can be a challenge due to the current set of keywords, as defined in the Java SE 12 Java Language Specification.
The goals of implementing hyphenated keywords, as specified in JDK-8223002, are:
- Explore the syntactic options open to Java language designers for denoting new features.
- Solve the perpetual problem of keyword tokens being so scarce and expensive that language designers have to constrain or corrupt the Java programming model to fit the keywords available.
- Advise language designers on the style of keyword suited to different kinds of features.
Several techniques have been used over the years to evolve the language: [a] eminent domain - reclassify an identifier as a keyword (such as assert in Java 1.4 and enum in Java 1.5); [b] overload - reuse an existing keyword for a new feature; [c] distort - create a syntax using an existing keyword (such as @interface); [d] smoke and mirrors - and create the illusion of a new keyword used in a new context (such as var with limited to local variables). These techniques have all been problematic in some way. For example, the addition of assert as a keyword broke almost every testing framework.
The hyphenated keyword concept would complement these existing techniques and would use a combination of existing classic and/or contextual keywords. Classic keywords are “a sequence of Java letters that is always tokenized as a keyword, never as an identifier.” Contextual keywords are “a sequence of Java letters that is tokenized as a keyword in certain contexts but as an identifier in all other contexts.” Examples of potential hyphenated classic keywords would include: non-final, break-with, and value-class. Examples of potential hyphenated contextual keywords would include: non-null, read-only, and eventually-true.
One of the challenges implementing hyphenated keywords involves how an arbitrary lookahead or fixed lookahead lexer should parse an expression such as a-b as a three tokens (identifier, operator, identifier) or a hyphenated keyword.
An OpenJDK e-mail conversation earlier this year proposed introducing the break-with hyphenated keyword for JDK 13 (scheduled to be released in September 2019). However, it was ultimately decided to drop the break-with keyword in favor of a new keyword, yield and to re-preview switch expressions. The eventual finalization of the new switch expression construct should clear the way to introducing the pattern matching concept that has been under discussion for almost two years.
Goetz and Buckley commented on the pros and cons of the hyphenated keyword concept, writing:
Leaving a feature out of Java for reasons of simplicity is fine; leaving it out because there is no way to denote the obvious semantics is not. This is a constant problem in evolving the language, and an ongoing tax paid by every Java developer.
One way to live without making new keywords is to stop evolving Java entirely. While there are some who think this is a fine idea, doing so because of the lack of available tokens would be a silly reason. Java has a long life ahead, and Java developers are excited about new features that enable to them to write more expressive and reliable code.
Goetz spoke to InfoQ about these proposed hyphenated keywords:
InfoQ: What has been the community response for hyphenated keywords in the Java language?
Brian Goetz: As you might expect, it has covered the spectrum. Some were pleased to see the care that goes into questions of how to best evolve a mature language; others complained that we were philosophizing about angels on the head of a pin, and would have preferred we work on their favorite feature instead.
InfoQ: What is the likelihood that the existing techniques, “eminent domain,” “overload,” “distort” and “smoke and mirrors,” be used again to expand the set of keywords in the Java language if the hyphenated keyword proposal isn't accepted?
Goetz: I look at it as a menu of options, with an interesting and flexible new option having been added to the menu. Still, in any given situation, one of the other options may still be better. We're not ruling anything out.
InfoQ: As per JEP 354, the proposed break-with hyphenated keyword was dropped in favor of the new keyword, yield for JDK 13. What led to the decision not to use the first hyphenated keyword in the Java language?
Goetz: The leading hyphenated candidate was
break-with, but in focus groups with users, people still found it fussy and non-obvious. Hyphenated keywords are on the menu, but that doesn't mean we have to order them for every meal. Overall, people foundyieldmore natural (though, of course, some didn't like that as it reminds them of returning a value from a coroutine). It turned out thatyield, in the specific grammar position it was in, was a reasonable candidate for a contextual keyword (not all grammar positions are as amenable).
InfoQ: As Java evolves more quickly with the six-month release cycle, how likely will a hyphenated keyword be ultimately introduced into the Java language?
Goetz: When we need one, it's on the menu. It's likely that Sealed Types will require a way to opt out of sealing, such as
non-sealed.
InfoQ: What is the single most important take-home message you would like our readers to know about hyphenated keywords?
Goetz: That we're serious about balancing the need to evolve the language compatibility while keeping it readable.
Resources
- Brian Goetz Speaks to InfoQ on Pattern Matching for Java by InfoQ (September 17, 2017)
- break-with by Brian Goetz (January 17, 2019)
- Keyword Management for the Java Language by Alex Buckley (April 25, 2019)
