bump version

👮‍♀️ Add "police", "law" and "cop" as tags for 👮‍♀️

.gitignore

@@ -1,2 +1,9 @@
+/bin/*
+!/bin/gemoji
+.bundle
 .ruby-version
-db/NamesList.txt
+Gemfile.lock
+db/emoji-test.txt
+db/ucd.nounihan.grouped.xml
+images/unicode/*.png
+vendor/

.travis.yml

@@ -1,5 +1,9 @@
+sudo: false
+script: script/test
 rvm:
   - 1.9.3
-  - 2.1.2
+  - '2.1'
+  - '2.2'
+  - 2.3.0
 notifications:
   email: false

CONTRIBUTING.md

@@ -0,0 +1,54 @@
+Contributions to this project are [released](https://help.github.com/articles/github-terms-of-service/#6-contributions-under-repository-license) to the public under the [project's open source license](LICENSE).
+
+This project adheres to the [Open Code of Conduct][code-of-conduct]. By participating, you are expected to uphold this code.
+[code-of-conduct]: http://todogroup.org/opencodeofconduct/#gemoji/opensource@github.com
+
+Our emoji set is based off Apple's emoji character palette, plus some custom
+emoji such as :octocat: :shipit: :metal:.
+
+Some useful tools in development are:
+
+```
+script/bootstrap
+```
+
+Sets up the development environment. The prerequisites are:
+
+* Ruby 1.9+
+* Bundler
+
+```
+rake db:generate
+```
+
+On OS X, this will rebuild the `db/Category-Emoji.json` file from the system
+one, pulling in any new emoji that Apple may have added in the meantime.
+
+```
+script/test
+```
+
+Runs the test suite, including the integrity test where we assert that we have
+covered each of Apple's emoji.
+
+```
+script/regenerate
+```
+
+Rebuilds the `db/emoji.json` file which is our main list of emoji: their
+canonical representations, descriptions, aliases, and tags. This requires OS X
+because Safari is used in the process to verify which character render as emoji
+and which render as ordinary Unicode glyphs from the current font.
+
+```
+script/console
+```
+
+Opens `irb` console with gemoji library preloded for experimentation.
+
+```
+script/release
+```
+
+For maintainers only: after the gemspec has been edited, this commits the
+change, tags a release, and pushes it to both GitHub and RubyGems.org.

Gemfile

@@ -1,6 +1,6 @@
 source "https://rubygems.org"
 
-gem "rake"
-gem "minitest"
+gem "rake", "~> 10.3.2"
+gem "minitest", "~> 5.3.5"
 
 gemspec

Gemfile.lock

@@ -1,18 +0,0 @@
-PATH
-  remote: .
-  specs:
-    gemoji (2.0.1)
-
-GEM
-  remote: https://rubygems.org/
-  specs:
-    minitest (5.3.5)
-    rake (10.3.2)
-
-PLATFORMS
-  ruby
-
-DEPENDENCIES
-  gemoji!
-  minitest
-  rake

LICENSE

@@ -10,9 +10,6 @@ Copyright (c) 2013 id Software. All rights reserved.
 trollface
 Copyright (c) 2013 whynne@deviantart. All rights reserved.
 
-All other images
-Copyright (c) 2013 Apple Inc. All rights reserved.
-
 Source code:
 
 Copyright (c) 2013 GitHub, Inc.

MAINTAINING.md

@@ -1,27 +0,0 @@
-# Maintainers
-
-## Releasing a new gem
-
-If you are just adding new emoji or making a small fix, only increment the patch level "1.0.x". If you need to rename a ton of emojis or making any other radical (but still mostly backwards compatible changes), bump the minor version "1.x.x".
-
-### Make a release commit
-
-To prepare the release commit, edit the [gemoji.gemspec](https://github.com/github/gemoji/blob/master/gemoji.gemspec) `version` value. Then make a single commit with the description as "Gemoji 1.x.x". Finally, tag the commit with `v1.x.x`.
-
-Example commit https://github.com/github/gemoji/commit/v1.0.0
-
-```
-$ git ci -m "Gemoji 1.0.0"
-$ git tag v1.0.0
-$ git push
-$ git push --tags
-```
-
-### Publish the gem
-
-Build and push the new gem with
-
-```
-$ gem build gemoji.gemspec
-$ gem push gemoji-1.0.0.gem
-```

README.md

@@ -1,7 +1,8 @@
 gemoji
 ======
 
-Emoji images and names. See the LICENSE for copyright information.
+This library contains character information about native emoji, as well as image
+files for a few custom emoji.
 
 
 Installation
@@ -13,36 +14,19 @@ Add `gemoji` to your Gemfile.
 gem 'gemoji'
 ```
 
-**Sync images**
+### Extract images
 
-Images can be copied to your public directory with `rake emoji` in your app. This is the recommended approach since the images will be available at a consistent location. This works best with cached formatted user content generated by tools like [html-pipeline](https://github.com/jch/html-pipeline).
+To obtain image files as fallbacks for browsers and OS's that don't support
+emoji, run the `gemoji extract` command **on macOS Sierra or later**:
 
-``` ruby
-# Rakefile
-load 'tasks/emoji.rake'
+``` sh
+bundle exec gemoji extract public/images/emoji
 ```
 
-```
-$ rake emoji
-```
+This will extract images into filenames such as:
 
-**Assets Precompiling**
-
-If you must, you can manually add all the images to your asset load path.
-
-``` ruby
-# config/application.rb
-config.assets.paths << Emoji.images_path
-```
-
-Then have them compiled to public on deploy.
-
-``` ruby
-# config/application.rb
-config.assets.precompile << "emoji/**/*.png"
-```
-
-**WARNING** Since there are a ton of images, just adding the path may slow down other lookups if you aren't using it. Compiling all the emojis on deploy will add overhead to your deploy if even the images haven't changed. Theres just so many more superfluous files to iterate over. Also, the urls will be fingerprinted which may not be ideal for referencing from cached content.
+* `public/images/emoji/octocat.png`
+* `public/images/emoji/unicode/1f9c0.png` (the `:cheese:` emoji)
 
 
 Example Rails Helper
@@ -56,8 +40,8 @@ See the [Emoji cheat sheet](http://www.emoji-cheat-sheet.com) for more examples.
 module EmojiHelper
   def emojify(content)
     h(content).to_str.gsub(/:([\w+-]+):/) do |match|
-      if emoji = Emoji.find_by_alias($1) { nil }
-        %(<img alt="#$1" src="#{asset_path("emoji/#{emoji.image_filename}")}" style="vertical-align:middle" width="20" height="20" />)
+      if emoji = Emoji.find_by_alias($1)
+        %(<img alt="#$1" src="#{image_path("emoji/#{emoji.image_filename}")}" style="vertical-align:middle" width="20" height="20" />)
       else
         match
       end
@@ -107,6 +91,14 @@ emoji.image_filename #=> "music.png"
 As you create new emoji, you must ensure that you also create and put the images
 they reference by their `image_filename` to your assets directory.
 
+You can customize `image_filename` with:
+
+```ruby
+emoji = Emoji.create("music") do |char|
+  char.image_filename = "subdirectory/my_emoji.gif"
+end
+```
+
 For existing emojis, you can edit the list of aliases or add new tags in an edit block:
 
 ```ruby

Rakefile

@@ -9,7 +9,11 @@ end
 
 namespace :db do
   desc %(Generate Emoji data files needed for development)
-  task :generate => ['db/Category-Emoji.json', 'db/NamesList.txt']
+  task :generate => [
+    'db/Category-Emoji.json',
+    'db/ucd.nounihan.grouped.xml',
+    'db/emoji-test.txt',
+  ]
 
   desc %(Dump a list of supported Emoji with Unicode descriptions and aliases)
   task :dump => :generate do
@@ -17,13 +21,25 @@ namespace :db do
   end
 end
 
-emoji_plist = '/System/Library/Input Methods/CharacterPalette.app/Contents/Resources/Category-Emoji.plist'
-nameslist_url = 'http://www.unicode.org/Public/6.3.0/ucd/NamesList.txt'
-
 task 'db/Category-Emoji.json' do |t|
-  system "plutil -convert json -r '#{emoji_plist}' -o '#{t.name}'"
+  system 'plutil', '-convert', 'json', '-r',
+    '/System/Library/Input Methods/CharacterPalette.app/Contents/Resources/Category-Emoji.plist',
+    '-o', t.name
 end
 
-file 'db/NamesList.txt' do |t|
-  system "curl -fsSL '#{nameslist_url}' -o '#{t.name}'"
+file 'db/ucd.nounihan.grouped.xml' do
+  Dir.chdir('db') do
+    system 'curl', '-fsSLO', 'http://www.unicode.org/Public/9.0.0/ucdxml/ucd.nounihan.grouped.zip'
+    system 'unzip', '-q', 'ucd.nounihan.grouped.zip'
+    rm 'ucd.nounihan.grouped.zip'
+  end
+end
+
+file 'db/emoji-test.txt' do |t|
+  system 'curl', '-fsSL', 'http://unicode.org/Public/emoji/4.0/emoji-test.txt', '-o', t.name
+end
+
+directory 'images/unicode' do
+  require 'emoji/extractor'
+  Emoji::Extractor.new(64, Emoji.images_path).extract!
 end

bin/gemoji

@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+require 'emoji/cli'
+
+exit_code = Emoji::CLI.dispatch(ARGV)
+exit exit_code

db/Category-Emoji.json

@@ -4,40 +4,64 @@
       "CVDataTitle" : "EmojiCategory-People",
       "CVCategoryImage" : "Emoji-HumanImage",
       "CVCategoryData" : {
-        "CVSkipNullGlyphs" : true,
-        "Data" : "😄,😃,😀,😊,☺️,😉,😍,😘,😚,😗,😙,😜,😝,😛,😳,😁,😔,😌,😒,😞,😣,😢,😂,😭,😪,😥,😰,😅,😓,😩,😫,😨,😱,😠,😡,😤,😖,😆,😋,😷,😎,😴,😵,😲,😟,😦,😧,😈,👿,😮,😬,😐,😕,😯,😶,😇,😏,😑,👲,👳,👮,👷,💂,👶,👦,👧,👨,👩,👴,👵,👱,👼,👸,😺,😸,😻,😽,😼,🙀,😿,😹,😾,👹,👺,🙈,🙉,🙊,💀,👽,💩,🔥,✨,🌟,💫,💥,💢,💦,💧,💤,💨,👂,👀,👃,👅,👄,👍,👎,👌,👊,✊,✌️,👋,✋,👐,👆,👇,👉,👈,🙌,🙏,☝️,👏,💪,🚶,🏃,💃,👫,👪,👬,👭,💏,💑,👯,🙆,🙅,💁,🙋,💆,💇,💅,👰,🙎,🙍,🙇,🎩,👑,👒,👟,👞,👡,👠,👢,👕,👔,👚,👗,🎽,👖,👘,👙,💼,👜,👝,👛,👓,🎀,🌂,💄,💛,💙,💜,💚,❤️,💔,💗,💓,💕,💖,💞,💘,💌,💋,💍,💎,👤,👥,💬,👣,💭"
+        "CVSkipNullGlyphs" : false,
+        "Data" : "😀,😃,😄,😁,😆,😅,😂,🤣,☺️,😊,😇,🙂,🙃,😉,😌,😍,😘,😗,😙,😚,😋,😜,😝,😛,🤑,🤗,🤓,😎,🤡,🤠,😏,😒,😞,😔,😟,😕,🙁,☹️,😣,😖,😫,😩,😤,😠,😡,😶,😐,😑,😯,😦,😧,😮,😲,😵,😳,😱,😨,😰,😢,😥,🤤,😭,😓,😪,😴,🙄,🤔,🤥,😬,🤐,🤢,🤧,😷,🤒,🤕,😈,👿,👹,👺,💩,👻,💀,☠️,👽,👾,🤖,🎃,😺,😸,😹,😻,😼,😽,🙀,😿,😾,👐,🙌,👏,🙏,🤝,👍,👎,👊,✊,🤛,🤜,🤞,✌️,🤘,👌,👈,👉,👆,👇,☝️,✋,🤚,🖐,🖖,👋,🤙,💪,🖕,✍️,🤳,💅,💍,💄,💋,👄,👅,👂,👃,👣,👁,👀,🗣,👤,👥,👶,👦,👧,👨,👩,👱‍♀️,👱,👴,👵,👲,👳‍♀️,👳,👮‍♀️,👮,👷‍♀️,👷,💂‍♀️,💂,🕵️‍♀️,🕵️,👩‍⚕️,👨‍⚕️,👩‍🌾,👨‍🌾,👩‍🍳,👨‍🍳,👩‍🎓,👨‍🎓,👩‍🎤,👨‍🎤,👩‍🏫,👨‍🏫,👩‍🏭,👨‍🏭,👩‍💻,👨‍💻,👩‍💼,👨‍💼,👩‍🔧,👨‍🔧,👩‍🔬,👨‍🔬,👩‍🎨,👨‍🎨,👩‍🚒,👨‍🚒,👩‍✈️,👨‍✈️,👩‍🚀,👨‍🚀,👩‍⚖️,👨‍⚖️,🤶,🎅,👸,🤴,👰,🤵,👼,🤰,🙇‍♀️,🙇,💁,💁‍♂️,🙅,🙅‍♂️,🙆,🙆‍♂️,🙋,🙋‍♂️,🤦‍♀️,🤦‍♂️,🤷‍♀️,🤷‍♂️,🙎,🙎‍♂️,🙍,🙍‍♂️,💇,💇‍♂️,💆,💆‍♂️,🕴,💃,🕺,👯,👯‍♂️,🚶‍♀️,🚶,🏃‍♀️,🏃,👫,👭,👬,💑,👩‍❤️‍👩,👨‍❤️‍👨,💏,👩‍❤️‍💋‍👩,👨‍❤️‍💋‍👨,👪,👨‍👩‍👧,👨‍👩‍👧‍👦,👨‍👩‍👦‍👦,👨‍👩‍👧‍👧,👩‍👩‍👦,👩‍👩‍👧,👩‍👩‍👧‍👦,👩‍👩‍👦‍👦,👩‍👩‍👧‍👧,👨‍👨‍👦,👨‍👨‍👧,👨‍👨‍👧‍👦,👨‍👨‍👦‍👦,👨‍👨‍👧‍👧,👩‍👦,👩‍👧,👩‍👧‍👦,👩‍👦‍👦,👩‍👧‍👧,👨‍👦,👨‍👧,👨‍👧‍👦,👨‍👦‍👦,👨‍👧‍👧,👚,👕,👖,👔,👗,👙,👘,👠,👡,👢,👞,👟,👒,🎩,🎓,👑,⛑,🎒,👝,👛,👜,💼,👓,🕶,🌂,☂️"
       }
     },
     {
       "CVDataTitle" : "EmojiCategory-Nature",
       "CVCategoryImage" : "Emoji-NatureImage",
       "CVCategoryData" : {
-        "CVSkipNullGlyphs" : true,
-        "Data" : "🐶,🐺,🐱,🐭,🐹,🐰,🐸,🐯,🐨,🐻,🐷,🐽,🐮,🐗,🐵,🐒,🐴,🐑,🐘,🐼,🐧,🐦,🐤,🐥,🐣,🐔,🐍,🐢,🐛,🐝,🐜,🐞,🐌,🐙,🐚,🐠,🐟,🐬,🐳,🐋,🐄,🐏,🐀,🐃,🐅,🐇,🐉,🐎,🐐,🐓,🐕,🐖,🐁,🐂,🐲,🐡,🐊,🐫,🐪,🐆,🐈,🐩,🐾,💐,🌸,🌷,🍀,🌹,🌻,🌺,🍁,🍃,🍂,🌿,🌾,🍄,🌵,🌴,🌲,🌳,🌰,🌱,🌼,🌐,🌞,🌝,🌚,🌑,🌒,🌓,🌔,🌕,🌖,🌗,🌘,🌜,🌛,🌙,🌍,🌎,🌏,🌋,🌌,🌠,⭐️,☀️,⛅️,☁️,⚡️,☔️,❄️,⛄️,🌀,🌁,🌈,🌊"
+        "CVSkipNullGlyphs" : false,
+        "Data" : "🐶,🐱,🐭,🐹,🐰,🦊,🐻,🐼,🐨,🐯,🦁,🐮,🐷,🐽,🐸,🐵,🙈,🙉,🙊,🐒,🐔,🐧,🐦,🐤,🐣,🐥,🦆,🦅,🦉,🦇,🐺,🐗,🐴,🦄,🐝,🐛,🦋,🐌,🐚,🐞,🐜,🕷,🕸,🐢,🐍,🦎,🦂,🦀,🦑,🐙,🦐,🐠,🐟,🐡,🐬,🦈,🐳,🐋,🐊,🐆,🐅,🐃,🐂,🐄,🦌,🐪,🐫,🐘,🦏,🦍,🐎,🐖,🐐,🐏,🐑,🐕,🐩,🐈,🐓,🦃,🕊,🐇,🐁,🐀,🐿,🐾,🐉,🐲,🌵,🎄,🌲,🌳,🌴,🌱,🌿,☘️,🍀,🎍,🎋,🍃,🍂,🍁,🍄,🌾,💐,🌷,🌹,🥀,🌻,🌼,🌸,🌺,🌎,🌍,🌏,🌕,🌖,🌗,🌘,🌑,🌒,🌓,🌔,🌚,🌝,🌞,🌛,🌜,🌙,💫,⭐️,🌟,✨,⚡️,🔥,💥,☄️,☀️,🌤,⛅️,🌥,🌦,🌈,☁️,🌧,⛈,🌩,🌨,☃️,⛄️,❄️,🌬,💨,🌪,🌫,🌊,💧,💦,☔️"
       }
     },
     {
-      "CVDataTitle" : "EmojiCategory-Objects",
-      "CVCategoryImage" : "Emoji-ObjectsImage",
+      "CVDataTitle" : "EmojiCategory-Foods",
+      "CVCategoryImage" : "Emoji-FoodsImage",
       "CVCategoryData" : {
-        "CVSkipNullGlyphs" : true,
-        "Data" : "🎍,💝,🎎,🎒,🎓,🎏,🎆,🎇,🎐,🎑,🎃,👻,🎅,🎄,🎁,🎋,🎉,🎊,🎈,🎌,🔮,🎥,📷,📹,📼,💿,📀,💽,💾,💻,📱,☎️,📞,📟,📠,📡,📺,📻,🔊,🔉,🔈,🔇,🔔,🔕,📢,📣,⏳,⌛️,⏰,⌚️,🔓,🔒,🔏,🔐,🔑,🔎,💡,🔦,🔆,🔅,🔌,🔋,🔍,🛁,🛀,🚿,🚽,🔧,🔩,🔨,🚪,🚬,💣,🔫,🔪,💊,💉,💰,💴,💵,💷,💶,💳,💸,📲,📧,📥,📤,✉️,📩,📨,📯,📫,📪,📬,📭,📮,📦,📝,📄,📃,📑,📊,📈,📉,📜,📋,📅,📆,📇,📁,📂,✂️,📌,📎,✒️,✏️,📏,📐,📕,📗,📘,📙,📓,📔,📒,📚,📖,🔖,📛,🔬,🔭,📰,🎨,🎬,🎤,🎧,🎼,🎵,🎶,🎹,🎻,🎺,🎷,🎸,👾,🎮,🃏,🎴,🀄️,🎲,🎯,🏈,🏀,⚽️,⚾️,🎾,🎱,🏉,🎳,⛳️,🚵,🚴,🏁,🏇,🏆,🎿,🏂,🏊,🏄,🎣,☕️,🍵,🍶,🍼,🍺,🍻,🍸,🍹,🍷,🍴,🍕,🍔,🍟,🍗,🍖,🍝,🍛,🍤,🍱,🍣,🍥,🍙,🍘,🍚,🍜,🍲,🍢,🍡,🍳,🍞,🍩,🍮,🍦,🍨,🍧,🎂,🍰,🍪,🍫,🍬,🍭,🍯,🍎,🍏,🍊,🍋,🍒,🍇,🍉,🍓,🍑,🍈,🍌,🍐,🍍,🍠,🍆,🍅,🌽"
+        "CVSkipNullGlyphs" : false,
+        "Data" : "🍏,🍎,🍐,🍊,🍋,🍌,🍉,🍇,🍓,🍈,🍒,🍑,🍍,🥝,🥑,🍅,🍆,🥒,🥕,🌽,🌶,🥔,🍠,🌰,🥜,🍯,🥐,🍞,🥖,🧀,🥚,🍳,🥓,🥞,🍤,🍗,🍖,🍕,🌭,🍔,🍟,🥙,🌮,🌯,🥗,🥘,🍝,🍜,🍲,🍥,🍣,🍱,🍛,🍚,🍙,🍘,🍢,🍡,🍧,🍨,🍦,🍰,🎂,🍮,🍭,🍬,🍫,🍿,🍩,🍪,🥛,🍼,☕️,🍵,🍶,🍺,🍻,🥂,🍷,🥃,🍸,🍹,🍾,🥄,🍴,🍽"
+      }
+    },
+    {
+      "CVDataTitle" : "EmojiCategory-Activity",
+      "CVCategoryImage" : "Emoji-ActivityImage",
+      "CVCategoryData" : {
+        "CVSkipNullGlyphs" : false,
+        "Data" : "⚽️,🏀,🏈,⚾️,🎾,🏐,🏉,🎱,🏓,🏸,🥅,🏒,🏑,🏏,⛳️,🏹,🎣,🥊,🥋,⛸,🎿,⛷,🏂,🏋️‍♀️,🏋️,🤺,🤼‍♀️,🤼‍♂️,🤸‍♀️,🤸‍♂️,⛹️‍♀️,⛹️,🤾‍♀️,🤾‍♂️,🏌️‍♀️,🏌️,🏄‍♀️,🏄,🏊‍♀️,🏊,🤽‍♀️,🤽‍♂️,🚣‍♀️,🚣,🏇,🚴‍♀️,🚴,🚵‍♀️,🚵,🎽,🏅,🎖,🥇,🥈,🥉,🏆,🏵,🎗,🎫,🎟,🎪,🤹‍♀️,🤹‍♂️,🎭,🎨,🎬,🎤,🎧,🎼,🎹,🥁,🎷,🎺,🎸,🎻,🎲,🎯,🎳,🎮,🎰"
       }
     },
     {
       "CVDataTitle" : "EmojiCategory-Places",
       "CVCategoryImage" : "Emoji-PlacesImage",
       "CVCategoryData" : {
-        "CVSkipNullGlyphs" : true,
-        "Data" : "🏠,🏡,🏫,🏢,🏣,🏥,🏦,🏪,🏩,🏨,💒,⛪️,🏬,🏤,🌇,🌆,🏯,🏰,⛺️,🏭,🗼,🗾,🗻,🌄,🌅,🌃,🗽,🌉,🎠,🎡,⛲️,🎢,🚢,⛵️,🚤,🚣,⚓️,🚀,✈️,💺,🚁,🚂,🚊,🚉,🚞,🚆,🚄,🚅,🚈,🚇,🚝,🚋,🚃,🚎,🚌,🚍,🚙,🚘,🚗,🚕,🚖,🚛,🚚,🚨,🚓,🚔,🚒,🚑,🚐,🚲,🚡,🚟,🚠,🚜,💈,🚏,🎫,🚦,🚥,⚠️,🚧,🔰,⛽️,🏮,🎰,♨️,🗿,🎪,🎭,📍,🚩,🇯🇵,🇰🇷,🇩🇪,🇨🇳,🇺🇸,🇫🇷,🇪🇸,🇮🇹,🇷🇺,🇬🇧"
+        "CVSkipNullGlyphs" : false,
+        "Data" : "🚗,🚕,🚙,🚌,🚎,🏎,🚓,🚑,🚒,🚐,🚚,🚛,🚜,🛴,🚲,🛵,🏍,🚨,🚔,🚍,🚘,🚖,🚡,🚠,🚟,🚃,🚋,🚞,🚝,🚄,🚅,🚈,🚂,🚆,🚇,🚊,🚉,🚁,🛩,✈️,🛫,🛬,🚀,🛰,💺,🛶,⛵️,🛥,🚤,🛳,⛴,🚢,⚓️,🚧,⛽️,🚏,🚦,🚥,🗺,🗿,🗽,⛲️,🗼,🏰,🏯,🏟,🎡,🎢,🎠,⛱,🏖,🏝,⛰,🏔,🗻,🌋,🏜,🏕,⛺️,🛤,🛣,🏗,🏭,🏠,🏡,🏘,🏚,🏢,🏬,🏣,🏤,🏥,🏦,🏨,🏪,🏫,🏩,💒,🏛,⛪️,🕌,🕍,🕋,⛩,🗾,🎑,🏞,🌅,🌄,🌠,🎇,🎆,🌇,🌆,🏙,🌃,🌌,🌉,🌁"
+      }
+    },
+    {
+      "CVDataTitle" : "EmojiCategory-Objects",
+      "CVCategoryImage" : "Emoji-ObjectsImage",
+      "CVCategoryData" : {
+        "CVSkipNullGlyphs" : false,
+        "Data" : "⌚️,📱,📲,💻,⌨️,🖥,🖨,🖱,🖲,🕹,🗜,💽,💾,💿,📀,📼,📷,📸,📹,🎥,📽,🎞,📞,☎️,📟,📠,📺,📻,🎙,🎚,🎛,⏱,⏲,⏰,🕰,⌛️,⏳,📡,🔋,🔌,💡,🔦,🕯,🗑,🛢,💸,💵,💴,💶,💷,💰,💳,💎,⚖️,🔧,🔨,⚒,🛠,⛏,🔩,⚙️,⛓,🔫,💣,🔪,🗡,⚔️,🛡,🚬,⚰️,⚱️,🏺,🔮,📿,💈,⚗️,🔭,🔬,🕳,💊,💉,🌡,🚽,🚰,🚿,🛁,🛀,🛎,🔑,🗝,🚪,🛋,🛏,🛌,🖼,🛍,🛒,🎁,🎈,🎏,🎀,🎊,🎉,🎎,🏮,🎐,✉️,📩,📨,📧,💌,📥,📤,📦,🏷,📪,📫,📬,📭,📮,📯,📜,📃,📄,📑,📊,📈,📉,🗒,🗓,📆,📅,📇,🗃,🗳,🗄,📋,📁,📂,🗂,🗞,📰,📓,📔,📒,📕,📗,📘,📙,📚,📖,🔖,🔗,📎,🖇,📐,📏,📌,📍,✂️,🖊,🖋,✒️,🖌,🖍,📝,✏️,🔍,🔎,🔏,🔐,🔒,🔓"
       }
     },
     {
       "CVDataTitle" : "EmojiCategory-Symbols",
       "CVCategoryImage" : "Emoji-SymbolImage",
       "CVCategoryData" : {
-        "CVSkipNullGlyphs" : true,
-        "Data" : "1️⃣,2️⃣,3️⃣,4️⃣,5️⃣,6️⃣,7️⃣,8️⃣,9️⃣,0️⃣,🔟,🔢,#️⃣,🔣,⬆️,⬇️,⬅️,➡️,🔠,🔡,🔤,↗️,↖️,↘️,↙️,↔️,↕️,🔄,◀️,▶️,🔼,🔽,↩️,↪️,ℹ️,⏪,⏩,⏫,⏬,⤵️,⤴️,🆗,🔀,🔁,🔂,🆕,🆙,🆒,🆓,🆖,📶,🎦,🈁,🈯️,🈳,🈵,🈴,🈲,🉐,🈹,🈺,🈶,🈚️,🚻,🚹,🚺,🚼,🚾,🚰,🚮,🅿️,♿️,🚭,🈷,🈸,🈂,Ⓜ️,🛂,🛄,🛅,🛃,🉑,㊙️,㊗️,🆑,🆘,🆔,🚫,🔞,📵,🚯,🚱,🚳,🚷,🚸,⛔️,✳️,❇️,❎,✅,✴️,💟,🆚,📳,📴,🅰,🅱,🆎,🅾,💠,➿,♻️,♈️,♉️,♊️,♋️,♌️,♍️,♎️,♏️,♐️,♑️,♒️,♓️,⛎,🔯,🏧,💹,💲,💱,©,®,™,❌,‼️,⁉️,❗️,❓,❕,❔,⭕️,🔝,🔚,🔙,🔛,🔜,🔃,🕛,🕧,🕐,🕜,🕑,🕝,🕒,🕞,🕓,🕟,🕔,🕠,🕕,🕖,🕗,🕘,🕙,🕚,🕡,🕢,🕣,🕤,🕥,🕦,✖️,➕,➖,➗,♠️,♥️,♣️,♦️,💮,💯,✔️,☑️,🔘,🔗,➰,〰,〽️,🔱,◼️,◻️,◾️,◽️,▪️,▫️,🔺,🔲,🔳,⚫️,⚪️,🔴,🔵,🔻,⬜️,⬛️,🔶,🔷,🔸,🔹"
+        "CVSkipNullGlyphs" : false,
+        "Data" : "❤️,💛,💚,💙,💜,🖤,💔,❣️,💕,💞,💓,💗,💖,💘,💝,💟,☮️,✝️,☪️,🕉,☸️,✡️,🔯,🕎,☯️,☦️,🛐,⛎,♈️,♉️,♊️,♋️,♌️,♍️,♎️,♏️,♐️,♑️,♒️,♓️,🆔,⚛️,🉑,☢️,☣️,📴,📳,🈶,🈚️,🈸,🈺,🈷️,✴️,🆚,💮,🉐,㊙️,㊗️,🈴,🈵,🈹,🈲,🅰️,🅱️,🆎,🆑,🅾️,🆘,❌,⭕️,🛑,⛔️,📛,🚫,💯,💢,♨️,🚷,🚯,🚳,🚱,🔞,📵,🚭,❗️,❕,❓,❔,‼️,⁉️,🔅,🔆,〽️,⚠️,🚸,🔱,⚜️,🔰,♻️,✅,🈯️,💹,❇️,✳️,❎,🌐,💠,Ⓜ️,🌀,💤,🏧,🚾,♿️,🅿️,🈳,🈂️,🛂,🛃,🛄,🛅,🚹,🚺,🚼,🚻,🚮,🎦,📶,🈁,🔣,ℹ️,🔤,🔡,🔠,🆖,🆗,🆙,🆒,🆕,🆓,0️⃣,1️⃣,2️⃣,3️⃣,4️⃣,5️⃣,6️⃣,7️⃣,8️⃣,9️⃣,🔟,🔢,#️⃣,*️⃣,▶️,⏸,⏯,⏹,⏺,⏭,⏮,⏩,⏪,⏫,⏬,◀️,🔼,🔽,➡️,⬅️,⬆️,⬇️,↗️,↘️,↙️,↖️,↕️,↔️,↪️,↩️,⤴️,⤵️,🔀,🔁,🔂,🔄,🔃,🎵,🎶,➕,➖,➗,✖️,💲,💱,™️,©️,®️,〰️,➰,➿,🔚,🔙,🔛,🔝,🔜,✔️,☑️,🔘,⚪️,⚫️,🔴,🔵,🔺,🔻,🔸,🔹,🔶,🔷,🔳,🔲,▪️,▫️,◾️,◽️,◼️,◻️,⬛️,⬜️,🔈,🔇,🔉,🔊,🔔,🔕,📣,📢,👁‍🗨,💬,💭,🗯,♠️,♣️,♥️,♦️,🃏,🎴,🀄️,🕐,🕑,🕒,🕓,🕔,🕕,🕖,🕗,🕘,🕙,🕚,🕛,🕜,🕝,🕞,🕟,🕠,🕡,🕢,🕣,🕤,🕥,🕦,🕧"
+      }
+    },
+    {
+      "CVDataTitle" : "EmojiCategory-Flags",
+      "CVCategoryImage" : "Emoji-FlagsImage",
+      "CVCategoryData" : {
+        "CVSkipNullGlyphs" : false,
+        "Data" : "🏳️,🏴,🏁,🚩,🏳️‍🌈,🇦🇫,🇦🇽,🇦🇱,🇩🇿,🇦🇸,🇦🇩,🇦🇴,🇦🇮,🇦🇶,🇦🇬,🇦🇷,🇦🇲,🇦🇼,🇦🇺,🇦🇹,🇦🇿,🇧🇸,🇧🇭,🇧🇩,🇧🇧,🇧🇾,🇧🇪,🇧🇿,🇧🇯,🇧🇲,🇧🇹,🇧🇴,🇧🇶,🇧🇦,🇧🇼,🇧🇷,🇮🇴,🇻🇬,🇧🇳,🇧🇬,🇧🇫,🇧🇮,🇨🇻,🇰🇭,🇨🇲,🇨🇦,🇮🇨,🇰🇾,🇨🇫,🇹🇩,🇨🇱,🇨🇳,🇨🇽,🇨🇨,🇨🇴,🇰🇲,🇨🇬,🇨🇩,🇨🇰,🇨🇷,🇨🇮,🇭🇷,🇨🇺,🇨🇼,🇨🇾,🇨🇿,🇩🇰,🇩🇯,🇩🇲,🇩🇴,🇪🇨,🇪🇬,🇸🇻,🇬🇶,🇪🇷,🇪🇪,🇪🇹,🇪🇺,🇫🇰,🇫🇴,🇫🇯,🇫🇮,🇫🇷,🇬🇫,🇵🇫,🇹🇫,🇬🇦,🇬🇲,🇬🇪,🇩🇪,🇬🇭,🇬🇮,🇬🇷,🇬🇱,🇬🇩,🇬🇵,🇬🇺,🇬🇹,🇬🇬,🇬🇳,🇬🇼,🇬🇾,🇭🇹,🇭🇳,🇭🇰,🇭🇺,🇮🇸,🇮🇳,🇮🇩,🇮🇷,🇮🇶,🇮🇪,🇮🇲,🇮🇱,🇮🇹,🇯🇲,🇯🇵,🎌,🇯🇪,🇯🇴,🇰🇿,🇰🇪,🇰🇮,🇽🇰,🇰🇼,🇰🇬,🇱🇦,🇱🇻,🇱🇧,🇱🇸,🇱🇷,🇱🇾,🇱🇮,🇱🇹,🇱🇺,🇲🇴,🇲🇰,🇲🇬,🇲🇼,🇲🇾,🇲🇻,🇲🇱,🇲🇹,🇲🇭,🇲🇶,🇲🇷,🇲🇺,🇾🇹,🇲🇽,🇫🇲,🇲🇩,🇲🇨,🇲🇳,🇲🇪,🇲🇸,🇲🇦,🇲🇿,🇲🇲,🇳🇦,🇳🇷,🇳🇵,🇳🇱,🇳🇨,🇳🇿,🇳🇮,🇳🇪,🇳🇬,🇳🇺,🇳🇫,🇲🇵,🇰🇵,🇳🇴,🇴🇲,🇵🇰,🇵🇼,🇵🇸,🇵🇦,🇵🇬,🇵🇾,🇵🇪,🇵🇭,🇵🇳,🇵🇱,🇵🇹,🇵🇷,🇶🇦,🇷🇪,🇷🇴,🇷🇺,🇷🇼,🇧🇱,🇸🇭,🇰🇳,🇱🇨,🇵🇲,🇻🇨,🇼🇸,🇸🇲,🇸🇹,🇸🇦,🇸🇳,🇷🇸,🇸🇨,🇸🇱,🇸🇬,🇸🇽,🇸🇰,🇸🇮,🇸🇧,🇸🇴,🇿🇦,🇬🇸,🇰🇷,🇸🇸,🇪🇸,🇱🇰,🇸🇩,🇸🇷,🇸🇿,🇸🇪,🇨🇭,🇸🇾,🇹🇼,🇹🇯,🇹🇿,🇹🇭,🇹🇱,🇹🇬,🇹🇰,🇹🇴,🇹🇹,🇹🇳,🇹🇷,🇹🇲,🇹🇨,🇹🇻,🇺🇬,🇺🇦,🇦🇪,🇬🇧,🇺🇸,🇻🇮,🇺🇾,🇺🇿,🇻🇺,🇻🇦,🇻🇪,🇻🇳,🇼🇫,🇪🇭,🇾🇪,🇿🇲,🇿🇼"
       }
     }
   ],

db/aliases.applescript

@@ -0,0 +1,13 @@
+set jsCode to "document.getElementById('output').value"
+set json to missing value
+
+tell application "Safari"
+  repeat
+    set json to (do JavaScript jsCode in current tab of window 1)
+    if (json is not missing value) then exit repeat
+    delay 0.5
+  end repeat
+  close current tab of window 1
+end tell
+
+return json

db/aliases.html

@@ -11,77 +11,76 @@
 <textarea id="output" rows="50" cols="80"></textarea>
 
 <script>
-  const VARIATION_SELECTOR_15 = String.fromCharCode(0xfe0e);
-  const VARIATION_SELECTOR_16 = String.fromCharCode(0xfe0f);
+  const VARIATION_SELECTOR_16 = /\ufe0f$/
 
   function detectAliases(db) {
-    for (var i = 0; i < db.length; ++i) {
-      var emoji = db[i];
-      var raw = emoji.emoji;
-      if (!raw) {
-        continue;
-      }
+    const canvas = document.createElement('canvas')
 
-      var candidates = [];
+    for (const emoji of db) {
+      const raw = emoji.emoji
+      if (!raw) continue
 
-      if (raw.length === 1) {
-        candidates.push(raw + VARIATION_SELECTOR_15);
-        candidates.push(raw + VARIATION_SELECTOR_16);
-      } else if (raw[raw.length - 1] === VARIATION_SELECTOR_16) {
-        var base = raw.substr(0, raw.length - 1);
-        candidates.push(base);
-        candidates.push(base + VARIATION_SELECTOR_15);
-      }
-
-      var aliases = candidates.filter(isColorEmoji);
-      if (aliases.length) {
-        emoji.unicodes = aliases;
+      const candidates = [raw]
+      if (VARIATION_SELECTOR_16.test(raw)) {
+        candidates.unshift(raw.replace(VARIATION_SELECTOR_16, ''))
       } else {
-        delete emoji.unicodes;
+        candidates.push(`${raw}\u{fe0f}`)
+      }
+
+      const newRaw = candidates.filter(e => detectEmoji(canvas, e))[0]
+
+      if (newRaw && newRaw != raw) {
+        emoji.emoji = newRaw
+        console.info('new raw representation found for %s :%s:', raw, emoji.aliases[0])
+      }
+      if (!newRaw) {
+        console.warn('no raw representation for %s :%s:', raw, emoji.aliases[0])
       }
     }
 
-    dump(db);
+    dump(db)
   }
 
-  function isColorEmoji(candidate) {
-    // Draw the emoji twice using a different color each time. If the emoji
-    // draws as the same color regardless of what color we set, it's a color
-    // emoji.
-    return color(candidate, "#f00") === color(candidate, "#0f0");
-  }
+  function detectEmoji(canvas, emoji) {
+    const context = canvas.getContext('2d')
+    drawEmoji(context, emoji)
 
-  var canvas = document.createElement("canvas");
-  function color(emoji, rgb) {
-    var context = canvas.getContext("2d");
-    context.clearRect(0, 0, 32, 32);
-    context.fillStyle = rgb;
-    context.textBaseline = "top";
-    context.font = "32px Arial";
-    context.fillText(emoji, 0, 0);
-    var data = context.getImageData(0, 0, 32, 32).data;
-    for (var i = 0; i < data.length; i += 4) {
-      if (data[i] === 0 && data[i + 1] === 0 && data[i + 2] === 0) {
-        continue;
+    let supported = false
+    const data = context.getImageData(0, 16, 64, 1).data
+
+    for (let x = 0; x <= 64; x++) {
+      if (x <= 32) {
+        // character is supported if there are any colored pixels found
+        if (data[4*x] || data[4*x + 1] || data[4*x + 2]) supported = true
+      } else if (x >= 48 && data[4*x + 3] > 0) {
+        // however, if more than one character got rendered, treat as unsupported
+        supported = false
+        break
       }
-      return data[i].toString(16)
-        + data[i + 1].toString(16)
-        + data[i + 2].toString(16);
     }
-    return "no colored pixel found";
+
+    return supported
+  }
+
+  function drawEmoji(context, emoji) {
+    context.clearRect(0, 0, 400, 400)
+    context.fillStyle = '#000'
+    context.textBaseline = 'top'
+    context.font = '32px sans-serif, "Apple Color Emoji", "Segoe UI", "Segoe UI Emoji", "Segoe UI Symbol"'
+    context.fillText(emoji+'___', 0, 0)
   }
 
   function dump(db) {
-    var json = JSON.stringify(db, null, "  ")
+    var json = JSON.stringify(db, null, '  ')
       .replace(/^( +)(.+)\[\](,?)$/mg, "$1$2[\n$1]$3")
-      .replace(/,\n( *)  /g, "\n$1, ");
-    document.getElementById("output").value = json + "\n";
+      .replace(/,\n( *)  /g, "\n$1, ")
+    document.getElementById('output').value = `${json}\n`
   }
 
-  var xhr = new XMLHttpRequest;
+  var xhr = new XMLHttpRequest
   xhr.onload = function() {
-    detectAliases(JSON.parse(this.responseText));
+    detectAliases(JSON.parse(xhr.responseText))
   };
-  xhr.open("GET", "emoji.json", false);
-  xhr.send(null);
+  xhr.open('GET', 'emoji.json', false)
+  xhr.send(null)
 </script>

db/dump.rb

@@ -1,28 +1,57 @@
 require 'emoji'
 require 'json'
-
-names_list = File.expand_path('../NamesList.txt', __FILE__)
+require 'rexml/document'
 
 class UnicodeCharacter
-  attr_reader :code, :description, :aliases
+  attr_reader :code, :description, :version, :aliases
 
-  @index = {}
-  class << self
-    attr_reader :index
+  class CharListener
+    CHAR_TAG = "char".freeze
 
-    def fetch(code, *args, &block)
-      code = code.to_s(16).rjust(4, '0') if code.is_a?(Integer)
-      index.fetch(code, *args, &block)
+    def self.parse(io, &block)
+      REXML::Document.parse_stream(io, self.new(&block))
     end
+
+    def initialize(&block)
+      @callback = block
+    end
+
+    def tag_start(name, attributes)
+      if CHAR_TAG == name
+        @callback.call(
+          attributes.fetch("cp") { return },
+          attributes.fetch("na") { return },
+          attributes.fetch("age", nil),
+        )
+      end
+    end
+
+    def method_missing(*) end
   end
 
-  def initialize(code, description)
+  def self.index
+    return @index if defined? @index
+    @index = {}
+    File.open(File.expand_path('../ucd.nounihan.grouped.xml', __FILE__)) do |source|
+      CharListener.parse(source) do |char, desc, age|
+        uc = UnicodeCharacter.new(char, desc, age)
+        @index[uc.code] = uc
+      end
+    end
+    @index
+  end
+
+  def self.fetch(code)
+    code = code.to_s(16).rjust(4, '0') if code.is_a?(Integer)
+    self.index.fetch(code)
+  end
+
+  def initialize(code, description, version)
     @code = code.downcase
     @description = description.downcase
+    @version = version
     @aliases = []
     @references = []
-
-    self.class.index[@code] = self
   end
 
   def add_alias(string)
@@ -34,44 +63,74 @@ class UnicodeCharacter
   end
 end
 
-char = nil
-
-File.foreach(names_list) do |line|
-  case line
-  when /^[A-F0-9]{4,5}\t/
-    code, desc = line.chomp.split("\t", 2)
-    codepoint = code.hex
-    char = UnicodeCharacter.new(code, desc)
-  when /^\t= /
-    char.add_alias($')
-  when /^\tx .+ - ([A-F0-9]{4,5})\)$/
-    char.add_reference($1)
+unless $stdin.tty?
+  codepoints = STDIN.read.chomp.codepoints.map { |code|
+    UnicodeCharacter.fetch(code)
+  }
+  codepoints.each do |char|
+    printf "%5s: %s", char.code.upcase, char.description
+    printf " (%s)", char.version if char.version
+    puts
   end
+  exit
 end
 
 trap(:PIPE) { abort }
 
-items = []
-variation = "\u{fe0f}".freeze
+normalize = -> (raw) {
+  raw.sub(Emoji::VARIATION_SELECTOR_16, '')
+}
 
-
-for emoji in Emoji.all
-  unicodes = emoji.unicode_aliases.dup
-
-  item = {}
-
-  unless emoji.custom?
-    variation_codepoint = variation.codepoints[0]
-    chars = emoji.raw.codepoints.map { |code| UnicodeCharacter.fetch(code) unless code == variation_codepoint }.compact
-    item[:emoji] = unicodes.shift
-    item[:unicodes] = unicodes if unicodes.any?
-    item[:description] = chars.map(&:description).join(' + ')
+emojidesc = {}
+File.open(File.expand_path('../emoji-test.txt', __FILE__)) do |file|
+  file.each do |line|
+    next if line =~ /^(#|$)/
+    line = line.chomp.split('# ', 2)[1]
+    emoji, description = line.split(' ', 2)
+    emojidesc[normalize.(emoji)] = description
   end
+end
 
-  item[:aliases] = emoji.aliases
-  item[:tags] = emoji.tags
+items = []
 
-  items << item
+for category, emojis in Emoji.apple_palette
+  for raw in emojis
+    emoji = Emoji.find_by_unicode(raw)
+    unicode_version = emoji ? emoji.unicode_version : ''
+    ios_version = emoji ? emoji.ios_version : ''
+
+    unless raw.include?(Emoji::ZERO_WIDTH_JOINER)
+      uchar = UnicodeCharacter.fetch(raw.codepoints[0])
+      unicode_version = uchar.version unless uchar.version.nil?
+    end
+
+    description = emojidesc.fetch(normalize.(raw))
+
+    if unicode_version == ''
+      warn "#{description} (#{raw}) doesn't have Unicode version"
+    end
+
+    if ios_version == ''
+      ios_version = '10.2'
+    end
+
+    items << {
+      emoji: raw,
+      description: description,
+      category: category,
+      aliases: emoji ? emoji.aliases : [description.gsub(/\W+/, '_').downcase],
+      tags: emoji ? emoji.tags : [],
+      unicode_version: unicode_version,
+      ios_version: ios_version,
+    }
+  end
+end
+
+for emoji in Emoji.all.select(&:custom?)
+  items << {
+    aliases: emoji.aliases,
+    tags: emoji.tags,
+  }
 end
 
 puts JSON.pretty_generate(items)

db/index.html

@@ -66,7 +66,7 @@ li > span {
       if (emoji.emoji) els[0].textContent = emoji.emoji
       else {
         var img = document.createElement('img')
-        img.src = "../images/emoji/" + emoji.aliases[0] + ".png"
+        img.src = "../images/" + emoji.aliases[0] + ".png"
         els[0].appendChild(img)
       }
       els[1].textContent = emoji.description || ''

gemoji.gemspec

@@ -1,8 +1,9 @@
 Gem::Specification.new do |s|
   s.name    = "gemoji"
-  s.version = "2.0.1"
-  s.summary = "Emoji conversion and image assets"
-  s.description = "Image assets and character information for emoji."
+  s.version = "3.0.1"
+  s.summary = "Emoji library"
+  s.description = "Character information and metadata for standard and custom emoji."
+  s.executables = ["gemoji"]
 
   s.required_ruby_version = '> 1.9'
 
@@ -13,9 +14,10 @@ Gem::Specification.new do |s|
 
   s.files = Dir[
     "README.md",
-    "images/**/*.png",
+    "bin/gemoji",
+    "images/*.png",
+    "db/Category-Emoji.json",
     "db/emoji.json",
     "lib/**/*.rb",
-    "lib/tasks/*.rake"
   ]
 end