Compare commits

...

70 Commits

Author SHA1 Message Date
Mislav Marohnić
0a24792305 gemoji 4.0.0.pre0 2019-07-05 10:32:29 +02:00
Mislav Marohnić
d48b0d90ac Merge pull request #164 from github/mislav/text-glyphs
Prefer `emoji.raw` without VARIATION SELECTOR 16
2019-07-03 20:11:32 +02:00
Mislav Marohnić
37b14e495b Ensure that emoji can't be looked up by their text glyph 2019-07-03 18:26:48 +02:00
Mislav Marohnić
fd84af55cf Regenerate emoji.json with shorter sequences 2019-07-03 17:57:25 +02:00
Mislav Marohnić
b37659ccd8 Use shortest possible sequence as emoji.raw representation 2019-07-03 17:57:25 +02:00
Mislav Marohnić
5766d15e27 Explicitly list glyphs to render as text when no VARIATION_SELECTOR_16 2019-07-03 17:37:56 +02:00
Mislav Marohnić
03dea3bd7d Merge pull request #161 from jhawthorn/dedup
Freeze and dedup strings
2019-07-03 15:34:12 +02:00
John Hawthorn
69009d4c1e Freeze and dedup strings 2019-05-15 16:16:37 -07:00
Mislav Marohnić
22b920f8bd Merge pull request #158 from github/rm-custom-emoji
Remove custom emoji
2019-05-06 18:47:57 +02:00
Mislav Marohnić
55a00803d8 Merge pull request #159 from github/unicode-12
Add Unicode 12 emoji
2019-05-05 03:47:38 +02:00
Mislav Marohnić
6f5518a9ed Add new Unicode 12 emoji 2019-05-05 03:26:18 +02:00
Mislav Marohnić
ff9b85268e Reorder existing emoji based on Unicode 12 2019-05-05 03:13:27 +02:00
Mislav Marohnić
c8cebe22eb Ensure that dump.rb accounts for all existing emoji.json entries 2019-05-05 03:12:43 +02:00
Mislav Marohnić
28a179180b Skip emoji-test.txt entries that are variations of skin tones 2019-05-05 03:11:46 +02:00
Mislav Marohnić
b801800935 Pull in emoji-test.txt for Unicode 12 2019-05-05 03:04:09 +02:00
Mislav Marohnić
16c382b744 Remove obsolete information 2019-05-05 02:28:19 +02:00
Mislav Marohnić
0f9400398e 🔥 extractor logic
We no longer ship any images to extract.
2019-05-05 02:28:19 +02:00
Mislav Marohnić
040aebef64 🔥 custom emoji
On GitHub.com, people predominantly use `:octocat:` and `:shipit:`, but
other custom emoji not as much. These two most used custom emoji also
happen to be specific to the GitHub brand, so it makes more sense that
we add them as custom emoji in the app itself rather than shipping them
with the gemoji library.
2019-05-05 02:28:19 +02:00
Mislav Marohnić
4b2d44d272 🔥 integrity tests
These were tied to the logic that extracts PNG images from Apple font
2019-05-05 02:02:03 +02:00
Mislav Marohnić
987463b50b 🔥 obsolete scripts 2019-05-05 01:58:35 +02:00
Mislav Marohnić
3bc9fe0694 Merge pull request #155 from github/no-apple
Deprecate Apple Palette and upgrade to Emoji 11.
2019-05-05 01:47:18 +02:00
Mislav Marohnić
e55d1457ab Vendor emoji-test.txt from Unicode.org so it can be used in tests 2019-05-05 01:27:01 +02:00
Mislav Marohnić
444ad5b1f4 Fix tests 2019-05-05 01:05:06 +02:00
Mislav Marohnić
e2911b66f2 Generate sequences with alternate VARIATION_SELECTOR_16 positioning 2019-05-05 01:05:06 +02:00
Mislav Marohnić
3e67cc2a60 Regenerate emoji.json 2019-05-05 01:05:06 +02:00
Mislav Marohnić
e5eb1e8bcf Cleanup in constants 2019-05-04 17:50:33 +02:00
Mislav Marohnić
f00846a597 Avoid reusing existing emoji in dump.rb 2019-05-04 17:49:45 +02:00
Mislav Marohnić
4301394495 Symbolize keys when parsing data file 2019-05-02 15:08:04 +02:00
Mislav Marohnić
294b5c4c5c Clean up hacks around gendered emoji 2019-05-02 14:54:06 +02:00
Mislav Marohnić
6ac95753f0 Remove palette method
This represented Apple's palette of characters before, but now has no
purpose since the `Emoji.all` list plus categories represents the
character palette recommendation from Unicode.
2019-05-02 14:52:12 +02:00
Mislav Marohnić
3e7a9bebe2 Parse emoji-test.txt from Unicode 2019-05-02 14:30:27 +02:00
Kristján Oddsson
70b65c3b10 Unicode 11 emojis 2019-05-02 14:30:27 +02:00
Mislav Marohnić
d582f48cbf Merge pull request #137 from nicolasleger/patch-1
Test build on new Ruby versions
2019-05-02 10:36:17 +02:00
Mislav Marohnić
a3cefa60de Ruby versions in CI without patch numbers 2019-05-02 10:23:16 +02:00
Mislav Marohnić
3668746720 Merge pull request #156 from ashmaroli/interpolate-regex-once
Interpolate gender-variation regex just once
2019-05-02 10:20:46 +02:00
Mislav Marohnić
1a652be3dc Merge pull request #152 from ashmaroli/frozen-string-literals
Freeze string literals when using a modern Ruby
2019-05-02 09:51:51 +02:00
Mislav Marohnić
df9d9491e8 Merge pull request #154 from ashmaroli/module-private-constant
Use Module#private_constant to hide constants
2019-05-02 09:51:26 +02:00
Ashwin Maroli
667d62c1a6 Interpolate gender-variation regex just once 2019-04-28 11:59:19 +05:30
Kristján Oddsson
8d7c6fa4e3 bump version 2019-04-25 11:19:03 +01:00
Kristján Oddsson
21c4e27ee2 Merge pull request #151 from github/policewoman-is-also-a-cop
👮‍♀️ Add "police", "law" and "cop" as tags for 👮‍♀️
2019-04-25 11:18:16 +01:00
Ashwin Maroli
7c91775a0b Use Module#private_constant to hide constants 2019-04-22 15:50:47 +05:30
Ashwin Maroli
afc91a5235 Freeze string literals when using a modern Ruby 2019-04-22 14:47:59 +05:30
Kristján Oddsson
aed88bb347 add "police", "law" and "cop" as tags for 👮‍♀️ 2019-04-18 08:38:24 +01:00
Mike McQuaid
b04991b001 Merge pull request #140 from github/contributing-license
CONTRIBUTING: note license.
2018-02-16 09:30:59 +00:00
Mike McQuaid
8d6aff3fc6 CONTRIBUTING: note license.
This is to be more explicit about the contribution process and license.
2018-02-16 09:09:16 +00:00
Nicolas Leger
3f8feffc48 Test build on new Ruby versions 2018-01-08 10:23:45 +01:00
Joshua Peek
3f6057146c Merge pull request #125 from github/sass
Add "sassy" alias to information desk people
2017-05-22 09:32:14 -07:00
Joshua Peek
61e16ae891 Add "sassy" alias to information desk people 2017-05-20 19:47:16 -07:00
Mislav Marohnić
9b8ad85de0 Merge branch 'unicode-aliases'
Fixes #122
2017-04-21 16:32:11 +02:00
Mislav Marohnić
09125ab983 Fix rendering of custom emoji in db/index.html 2017-04-21 16:31:32 +02:00
Mislav Marohnić
b9ce241bf1 Regenerate emoji.json with minimal raw representations
If a raw representation of emoji includes VARIATION SELECTOR 16, but a
browser would render it as a color emoji even without that character,
drop the VARIATION SELECTOR 16 from the raw representation.
2017-04-21 16:31:32 +02:00
Mislav Marohnić
ed954928d7 Improve script for detecting supported unicode aliases 2017-04-21 16:31:31 +02:00
Mislav Marohnić
7858e79566 Merge pull request #116 from skunkmb/master
Add a laugh alias and a hooray alias to 😄 and 🎉
2017-02-16 12:00:55 +01:00
Skunk
7bdd3f657f Use tags instead of aliases for laugh and hooray
As mentioned by @mislav in github/gemoji#116, too many aliases can be
tricky, but tags don’t need to be unique. Therefore, replace new aliases
with new tags for the recently changed 😄 and 🎉 emoji.
2017-02-15 16:23:33 -08:00
Skunk
68af9e3320 Add a laugh alias and a hooray alias to 😄 and 🎉
In the GitHub Reactions menu, there are six emoji: 👍, 👎, 😄, 🎉, 😕, and
❤️. However, two out of the six (😄 and 🎉) have descriptions (“Laugh”
and “Hooray”) that do not actually correspond to Gemoji aliases. Add new
aliases to those two Gemoji so that they directly correspond to the
GitHub Reactions.
2017-02-12 08:41:46 -08:00
Mislav Marohnić
7d27517382 gemoji 3.0.0 2016-12-21 17:14:05 +01:00
Mislav Marohnić
53325ab398 Allow specifying image size to extract 2016-12-21 17:12:58 +01:00
Mislav Marohnić
09f66fc38a gemoji 3.0.0.rc2 2016-12-20 17:36:21 +01:00
Mislav Marohnić
342b76f06e Merge branch 'ios-10.2' 2016-12-20 17:34:49 +01:00
Mislav Marohnić
6b556ba8b2 Update integrity test
These images are finally no longer identical.
2016-12-20 17:34:00 +01:00
Mislav Marohnić
74a321c07a Fix filling in emoji that historically had default gender
Now in iOS 10.2 there are gendered emoji that don't have a default
genderless unicode variant.
2016-12-20 17:34:00 +01:00
Mislav Marohnić
7e10d1a38a Change egg in a pan to "fried_egg" since "egg" is now an emoji 2016-12-20 17:34:00 +01:00
Mislav Marohnić
8ff20db97f Add new iOS 10.2 emoji 2016-12-20 17:34:00 +01:00
Mislav Marohnić
492e18f849 Reorder existing emoji according to iOS 10.2 palette 2016-12-20 17:34:00 +01:00
Mislav Marohnić
cf852befd9 Improve dump script for new emoji 2016-12-20 17:33:52 +01:00
Mislav Marohnić
9d313238bd Remove Gemfile.lock from version control
I don't like that Bundler now makes Gemfile.lock change whenever I bump
up gemoji version. There aren't many dependencies, so a missing
Gemfile.lock won't slow down CI too much.
2016-12-20 15:15:28 +01:00
Alexander Kotov
840655ad6b Fix NoMethodError in multithreading
Fixes #109, closes #111
2016-12-20 15:13:35 +01:00
Mislav Marohnić
f394c85d80 Fix Gemfile.lock 2016-09-30 13:18:43 -04:00
Mislav Marohnić
42a606c804 Merge pull request #99 from mikeastock/fix-sierra-typo
Fix typo in README of Sierra
2016-09-30 08:17:27 -09:00
Michael Stock
21c4cfe247 Fix typo in README of Sierra 2016-09-29 23:14:17 -07:00
43 changed files with 22347 additions and 13603 deletions

7
.gitignore vendored
View File

@@ -1,7 +1,6 @@
/bin/*
!/bin/gemoji
.bundle
.ruby-version
db/NamesList.txt
images/unicode/*.png
vendor/
Gemfile.lock
vendor/*
!vendor/unicode-emoji-test.txt

View File

@@ -4,6 +4,9 @@ rvm:
- 1.9.3
- '2.1'
- '2.2'
- 2.3.0
- '2.3'
- '2.4'
- '2.5'
- '2.6'
notifications:
email: false

View File

@@ -1,8 +1,4 @@
This project adheres to the [Open Code of Conduct][code-of-conduct]. By participating, you are expected to uphold this code.
[code-of-conduct]: http://todogroup.org/opencodeofconduct/#gemoji/opensource@github.com
Our emoji set is based off Apple's emoji character palette, plus some custom
emoji such as :octocat: :shipit: :metal:.
Contributions to this project are [released](https://help.github.com/articles/github-terms-of-service/#6-contributions-under-repository-license) to the public under the [project's open source license](LICENSE).
Some useful tools in development are:
@@ -15,28 +11,11 @@ Sets up the development environment. The prerequisites are:
* Ruby 1.9+
* Bundler
```
rake db:generate
```
On OS X, this will rebuild the `db/Category-Emoji.json` file from the system
one, pulling in any new emoji that Apple may have added in the meantime.
```
script/test
```
Runs the test suite, including the integrity test where we assert that we have
covered each of Apple's emoji.
```
script/regenerate
```
Rebuilds the `db/emoji.json` file which is our main list of emoji: their
canonical representations, descriptions, aliases, and tags. This requires OS X
because Safari is used in the process to verify which character render as emoji
and which render as ordinary Unicode glyphs from the current font.
Runs the test suite.
```
script/console

View File

@@ -1,6 +1,6 @@
source "https://rubygems.org"
gem "rake"
gem "minitest"
gem "rake", "~> 10.3.2"
gem "minitest", "~> 5.3.5"
gemspec

View File

@@ -1,21 +0,0 @@
PATH
remote: .
specs:
gemoji (2.1.0)
GEM
remote: https://rubygems.org/
specs:
minitest (5.3.5)
rake (10.3.2)
PLATFORMS
ruby
DEPENDENCIES
gemoji!
minitest
rake
BUNDLED WITH
1.11.2

16
LICENSE
View File

@@ -1,18 +1,4 @@
octocat, squirrel, shipit
Copyright (c) 2013 GitHub Inc. All rights reserved.
bowtie, neckbeard, fu
Copyright (c) 2013 37signals, LLC. All rights reserved.
feelsgood, finnadie, goberserk, godmode, hurtrealbad, rage 1-4, suspect
Copyright (c) 2013 id Software. All rights reserved.
trollface
Copyright (c) 2013 whynne@deviantart. All rights reserved.
Source code:
Copyright (c) 2013 GitHub, Inc.
Copyright (c) 2019 GitHub, Inc.
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation

View File

@@ -1,8 +1,7 @@
gemoji
======
This library contains character information about native emoji, as well as image
files for a few custom emoji.
This library contains character information about native emojis.
Installation
@@ -14,20 +13,6 @@ Add `gemoji` to your Gemfile.
gem 'gemoji'
```
### Extract images
To obtain image files as fallbacks for browsers and OS's that don't support
emoji, run the `gemoji extract` command **on macOS Seirra or later**:
``` sh
bundle exec gemoji extract public/images/emoji
```
This will extract images into filenames such as:
* `public/images/emoji/octocat.png`
* `public/images/emoji/unicode/1f9c0.png` (the `:cheese:` emoji)
Example Rails Helper
--------------------

View File

@@ -9,7 +9,9 @@ end
namespace :db do
desc %(Generate Emoji data files needed for development)
task :generate => ['db/Category-Emoji.json', 'db/NamesList.txt']
task :generate => [
'vendor/unicode-emoji-test.txt',
]
desc %(Dump a list of supported Emoji with Unicode descriptions and aliases)
task :dump => :generate do
@@ -17,18 +19,6 @@ namespace :db do
end
end
emoji_plist = '/System/Library/Input Methods/CharacterPalette.app/Contents/Resources/Category-Emoji.plist'
nameslist_url = 'http://www.unicode.org/Public/6.3.0/ucd/NamesList.txt'
task 'db/Category-Emoji.json' do |t|
system "plutil -convert json -r '#{emoji_plist}' -o '#{t.name}'"
end
file 'db/NamesList.txt' do |t|
system "curl -fsSL '#{nameslist_url}' -o '#{t.name}'"
end
directory 'images/unicode' do
require 'emoji/extractor'
Emoji::Extractor.new(64, Emoji.images_path).extract!
file 'vendor/unicode-emoji-test.txt' do |t|
system 'curl', '-fsSL', 'http://unicode.org/Public/emoji/12.0/emoji-test.txt', '-o', t.name
end

View File

@@ -1,5 +0,0 @@
#!/usr/bin/env ruby
require 'emoji/cli'
exit_code = Emoji::CLI.dispatch(ARGV)
exit exit_code

View File

@@ -1,71 +0,0 @@
{
"EmojiDataArray" : [
{
"CVDataTitle" : "EmojiCategory-People",
"CVCategoryImage" : "Emoji-HumanImage",
"CVCategoryData" : {
"CVSkipNullGlyphs" : false,
"Data" : "😀,😬,😁,😂,😃,😄,😅,😆,😇,😉,😊,🙂,🙃,☺️,😋,😌,😍,😘,😗,😙,😚,😜,😝,😛,🤑,🤓,😎,🤗,😏,😶,😐,😑,😒,🙄,🤔,😳,😞,😟,😠,😡,😔,😕,🙁,☹️,😣,😖,😫,😩,😤,😮,😱,😨,😰,😯,😦,😧,😢,😥,😪,😓,😭,😵,😲,🤐,😷,🤒,🤕,😴,💤,💩,😈,👿,👹,👺,👻,💀,☠️,👽,👾,🤖,😺,😸,😹,😻,😼,😽,🙀,😿,😾,🙌,👏,👍,👎,👊,✊,👋,👈,👉,👆,👇,👌,☝️,✌️,✋,🖐,👐,💪,🙏,🖖,🤘,🖕,✍️,💅,👄,👅,👂,👃,👁,👀,🗣,👤,👥,👶,👦,👧,👨,👩,👱‍♀️,👱,👴,👵,👲,👳‍♀️,👳,👮‍♀️,👮,👷‍♀️,👷,💂‍♀️,💂,🕵️‍♀️,🕵️,🎅,👸,👰,👼,🙇‍♀️,🙇,💁,💁‍♂️,🙅,🙅‍♂️,🙆,🙆‍♂️,🙋,🙋‍♂️,🙎,🙎‍♂️,🙍,🙍‍♂️,💇,💇‍♂️,💆,💆‍♂️,💃,👯,👯‍♂️,🚶‍♀️,🚶,🏃‍♀️,🏃,👫,👭,👬,💑,👩‍❤️‍👩,👨‍❤️‍👨,💏,👩‍❤️‍💋‍👩,👨‍❤️‍💋‍👨,👪,👨‍👩‍👧,👨‍👩‍👧‍👦,👨‍👩‍👦‍👦,👨‍👩‍👧‍👧,👩‍👩‍👦,👩‍👩‍👧,👩‍👩‍👧‍👦,👩‍👩‍👦‍👦,👩‍👩‍👧‍👧,👨‍👨‍👦,👨‍👨‍👧,👨‍👨‍👧‍👦,👨‍👨‍👦‍👦,👨‍👨‍👧‍👧,👩‍👦,👩‍👧,👩‍👧‍👦,👩‍👦‍👦,👩‍👧‍👧,👨‍👦,👨‍👧,👨‍👧‍👦,👨‍👦‍👦,👨‍👧‍👧,👚,👕,👖,👔,👗,👙,👘,💄,💋,👣,👠,👡,👢,👞,👟,👒,🎩,🎓,👑,⛑,🎒,👝,👛,👜,💼,👓,🕶,💍,🌂"
}
},
{
"CVDataTitle" : "EmojiCategory-Nature",
"CVCategoryImage" : "Emoji-NatureImage",
"CVCategoryData" : {
"CVSkipNullGlyphs" : false,
"Data" : "🐶,🐱,🐭,🐹,🐰,🐻,🐼,🐨,🐯,🦁,🐮,🐷,🐽,🐸,🐙,🐵,🙈,🙉,🙊,🐒,🐔,🐧,🐦,🐤,🐣,🐥,🐺,🐗,🐴,🦄,🐝,🐛,🐌,🐞,🐜,🕷,🦂,🦀,🐍,🐢,🐠,🐟,🐡,🐬,🐳,🐋,🐊,🐆,🐅,🐃,🐂,🐄,🐪,🐫,🐘,🐐,🐏,🐑,🐎,🐖,🐀,🐁,🐓,🦃,🕊,🐕,🐩,🐈,🐇,🐿,🐾,🐉,🐲,🌵,🎄,🌲,🌳,🌴,🌱,🌿,☘,🍀,🎍,🎋,🍃,🍂,🍁,🌾,🌺,🌻,🌹,🌷,🌼,🌸,💐,🍄,🌰,🎃,🐚,🕸,🌎,🌍,🌏,🌕,🌖,🌗,🌘,🌑,🌒,🌓,🌔,🌚,🌝,🌛,🌜,🌞,🌙,⭐️,🌟,💫,✨,☄️,☀️,🌤,⛅️,🌥,🌦,☁️,🌧,⛈,🌩,⚡️,🔥,💥,❄️,🌨,☃️,⛄️,🌬,💨,🌪,🌫,☂️,☔️,💧,💦,🌊"
}
},
{
"CVDataTitle" : "EmojiCategory-Foods",
"CVCategoryImage" : "Emoji-FoodsImage",
"CVCategoryData" : {
"CVSkipNullGlyphs" : false,
"Data" : "🍏,🍎,🍐,🍊,🍋,🍌,🍉,🍇,🍓,🍈,🍒,🍑,🍍,🍅,🍆,🌶,🌽,🍠,🍯,🍞,🧀,🍗,🍖,🍤,🍳,🍔,🍟,🌭,🍕,🍝,🌮,🌯,🍜,🍲,🍥,🍣,🍱,🍛,🍙,🍚,🍘,🍢,🍡,🍧,🍨,🍦,🍰,🎂,🍮,🍬,🍭,🍫,🍿,🍩,🍪,🍺,🍻,🍷,🍸,🍹,🍾,🍶,🍵,☕️,🍼,🍴,🍽"
}
},
{
"CVDataTitle" : "EmojiCategory-Activity",
"CVCategoryImage" : "Emoji-ActivityImage",
"CVCategoryData" : {
"CVSkipNullGlyphs" : false,
"Data" : "⚽️,🏀,🏈,⚾️,🎾,🏐,🏉,🎱,🏓,🏸,🏒,🏑,🏏,🏹,⛳️,🎣,⛸,🎿,⛷,🏂,🏋️‍♀️,🏋️,⛹️‍♀️,⛹️,🏌️‍♀️,🏌️,🏄‍♀️,🏄,🏊‍♀️,🏊,🚣‍♀️,🚣,🏇,🚴‍♀️,🚴,🚵‍♀️,🚵,🛀,🕴,🎗,🎽,🏅,🎖,🏆,🏵,🎯,🎫,🎟,🎭,🎨,🎪,🎬,🎤,🎧,🎼,🎹,🎷,🎺,🎸,🎻,🎮,🎰,🎲,🎳"
}
},
{
"CVDataTitle" : "EmojiCategory-Places",
"CVCategoryImage" : "Emoji-PlacesImage",
"CVCategoryData" : {
"CVSkipNullGlyphs" : false,
"Data" : "🚗,🚕,🚙,🚌,🚎,🏎,🚓,🚑,🚒,🚐,🚚,🚛,🚜,🏍,🚲,🚨,🚔,🚍,🚘,🚖,🚡,🚠,🚟,🚃,🚋,🚝,🚄,🚅,🚈,🚞,🚂,🚆,🚇,🚊,🚉,🚁,🛩,✈️,🛫,🛬,⛵️,🛥,🚤,⛴,🛳,🚀,🛰,💺,⚓️,🚧,⛽️,🚏,🚦,🚥,🗺,🚢,🎡,🎢,🎠,🏗,🌁,🗼,🏭,⛲️,🎑,⛰,🏔,🗻,🌋,🗾,🏕,⛺️,🏞,🛣,🛤,🌅,🌄,🏜,🏖,🏝,🌇,🌆,🏙,🌃,🌉,🌌,🌠,🎇,🎆,🌈,🏘,🏰,🏯,🏟,🗽,🏠,🏡,🏚,🏢,🏬,🏣,🏤,🏥,🏦,🏨,🏪,🏫,🏩,💒,🏛,⛪️,🕌,🕍,🕋,⛩"
}
},
{
"CVDataTitle" : "EmojiCategory-Objects",
"CVCategoryImage" : "Emoji-ObjectsImage",
"CVCategoryData" : {
"CVSkipNullGlyphs" : false,
"Data" : "⌚️,📱,📲,💻,⌨️,🖥,🖨,🖱,🖲,🕹,🗜,💽,💾,💿,📀,📼,📷,📸,📹,🎥,📽,🎞,📞,☎️,📟,📠,📺,📻,🎙,🎚,🎛,⏱,⏲,⏰,🕰,⏳,⌛️,📡,🔋,🔌,💡,🔦,🕯,🗑,🛢,💸,💵,💴,💶,💷,💰,💳,💎,⚖,🔧,🔨,⚒,🛠,⛏,🔩,⚙,⛓,🔫,💣,🔪,🗡,⚔,🛡,🚬,⚰,⚱,🏺,🔮,📿,💈,⚗,🔭,🔬,🕳,💊,💉,🌡,🚽,🚿,🛁,🛎,🔑,🗝,🚪,🛋,🛌,🛏,🖼,⛱,🗿,🛍,🎁,🎈,🎏,🎀,🎊,🎉,🎐,🏮,🎎,✉️,📩,📨,📧,💌,📥,📤,📦,🏷,🔖,📪,📫,📬,📭,📮,📯,📜,📃,📄,📑,📊,📈,📉,🗒,🗓,📆,📅,📇,🗃,🗳,🗄,📋,📁,📂,🗂,🗞,📰,📓,📔,📒,📕,📗,📘,📙,📚,📖,🔗,📎,🖇,📐,📏,✂️,📌,📍,🚩,🎌,🏳️,🏴,🏁,🏳️‍🌈,🖌,🖍,🖊,🖋,✒️,📝,✏️,🔏,🔐,🔒,🔓,🔍,🔎"
}
},
{
"CVDataTitle" : "EmojiCategory-Symbols",
"CVCategoryImage" : "Emoji-SymbolImage",
"CVCategoryData" : {
"CVSkipNullGlyphs" : false,
"Data" : "❤️,💛,💚,💙,💜,💔,❣️,💕,💞,💓,💗,💖,💘,💝,💟,☮️,✝️,☪️,🕉,☸️,✡️,🔯,🕎,☯️,☦️,🛐,⛎,♈️,♉️,♊️,♋️,♌️,♍️,♎️,♏️,♐️,♑️,♒️,♓️,🆔,⚛,🈳,🈹,☢️,☣️,📴,📳,🈶,🈚️,🈸,🈺,🈷️,✴️,🆚,🉑,💮,🉐,㊙️,㊗️,🈴,🈵,🈲,🅰️,🅱️,🆎,🆑,🅾️,🆘,⛔️,📛,🚫,❌,⭕️,💢,♨️,🚷,🚯,🚳,🚱,🔞,📵,❗️,❕,❓,❔,‼️,⁉️,💯,🔅,🔆,🔱,⚜,〽️,⚠️,🚸,🔰,♻️,🈯️,💹,❇️,✳️,❎,✅,🌐,Ⓜ️,💠,🌀,➿,🏧,🈂️,🛂,🛃,🛄,🛅,♿️,🚭,🚾,🅿️,🚰,🚹,🚺,🚼,🚻,🚮,🎦,📶,🈁,🔤,🔡,🔠,🔣,,🆖,🆗,🆙,🆒,🆕,🆓,0⃣,1⃣,2⃣,3⃣,4⃣,5⃣,6⃣,7⃣,8⃣,9⃣,🔟,🔢,#️⃣,*️⃣,▶️,⏸,⏯,⏹,⏺,⏭,⏮,⏩,⏪,⏫,⏬,◀️,🔼,🔽,➡️,⬅️,⬆️,⬇️,↗️,↘️,↙️,↖️,↕️,↔️,↪️,↩️,⤴️,⤵️,🔀,🔁,🔂,🔄,🔃,🎵,🎶,〰️,➰,✔️,,,➗,✖️,💲,💱,™️,©️,®️,🔚,🔙,🔛,🔝,🔜,☑️,🔘,⚪️,⚫️,🔴,🔵,🔺,🔻,🔸,🔹,🔶,🔷,🔳,🔲,▪️,▫️,◾️,◽️,◼️,◻️,⬛️,⬜️,🔇,🔈,🔉,🔊,🔕,🔔,📣,📢,👁‍🗨,💬,💭,🗯,🃏,🀄️,🎴,♠️,♣️,♥️,♦️,🕐,🕑,🕒,🕓,🕔,🕕,🕖,🕗,🕘,🕙,🕚,🕛,🕜,🕝,🕞,🕟,🕠,🕡,🕢,🕣,🕤,🕥,🕦,🕧"
}
},
{
"CVDataTitle" : "EmojiCategory-Flags",
"CVCategoryImage" : "Emoji-FlagsImage",
"CVCategoryData" : {
"CVSkipNullGlyphs" : false,
"Data" : "🇦🇫,🇦🇽,🇦🇱,🇩🇿,🇦🇸,🇦🇩,🇦🇴,🇦🇮,🇦🇶,🇦🇬,🇦🇷,🇦🇲,🇦🇼,🇦🇺,🇦🇹,🇦🇿,🇧🇸,🇧🇭,🇧🇩,🇧🇧,🇧🇾,🇧🇪,🇧🇿,🇧🇯,🇧🇲,🇧🇹,🇧🇴,🇧🇶,🇧🇦,🇧🇼,🇧🇷,🇮🇴,🇻🇬,🇧🇳,🇧🇬,🇧🇫,🇧🇮,🇨🇻,🇰🇭,🇨🇲,🇨🇦,🇮🇨,🇰🇾,🇨🇫,🇹🇩,🇨🇱,🇨🇳,🇨🇽,🇨🇨,🇨🇴,🇰🇲,🇨🇬,🇨🇩,🇨🇰,🇨🇷,🇭🇷,🇨🇺,🇨🇼,🇨🇾,🇨🇿,🇩🇰,🇩🇯,🇩🇲,🇩🇴,🇪🇨,🇪🇬,🇸🇻,🇬🇶,🇪🇷,🇪🇪,🇪🇹,🇪🇺,🇫🇰,🇫🇴,🇫🇯,🇫🇮,🇫🇷,🇬🇫,🇵🇫,🇹🇫,🇬🇦,🇬🇲,🇬🇪,🇩🇪,🇬🇭,🇬🇮,🇬🇷,🇬🇱,🇬🇩,🇬🇵,🇬🇺,🇬🇹,🇬🇬,🇬🇳,🇬🇼,🇬🇾,🇭🇹,🇭🇳,🇭🇰,🇭🇺,🇮🇸,🇮🇳,🇮🇩,🇮🇷,🇮🇶,🇮🇪,🇮🇲,🇮🇱,🇮🇹,🇨🇮,🇯🇲,🇯🇵,🇯🇪,🇯🇴,🇰🇿,🇰🇪,🇰🇮,🇽🇰,🇰🇼,🇰🇬,🇱🇦,🇱🇻,🇱🇧,🇱🇸,🇱🇷,🇱🇾,🇱🇮,🇱🇹,🇱🇺,🇲🇴,🇲🇰,🇲🇬,🇲🇼,🇲🇾,🇲🇻,🇲🇱,🇲🇹,🇲🇭,🇲🇶,🇲🇷,🇲🇺,🇾🇹,🇲🇽,🇫🇲,🇲🇩,🇲🇨,🇲🇳,🇲🇪,🇲🇸,🇲🇦,🇲🇿,🇲🇲,🇳🇦,🇳🇷,🇳🇵,🇳🇱,🇳🇨,🇳🇿,🇳🇮,🇳🇪,🇳🇬,🇳🇺,🇳🇫,🇲🇵,🇰🇵,🇳🇴,🇴🇲,🇵🇰,🇵🇼,🇵🇸,🇵🇦,🇵🇬,🇵🇾,🇵🇪,🇵🇭,🇵🇳,🇵🇱,🇵🇹,🇵🇷,🇶🇦,🇷🇪,🇷🇴,🇷🇺,🇷🇼,🇧🇱,🇸🇭,🇰🇳,🇱🇨,🇵🇲,🇻🇨,🇼🇸,🇸🇲,🇸🇹,🇸🇦,🇸🇳,🇷🇸,🇸🇨,🇸🇱,🇸🇬,🇸🇽,🇸🇰,🇸🇮,🇸🇧,🇸🇴,🇿🇦,🇬🇸,🇰🇷,🇸🇸,🇪🇸,🇱🇰,🇸🇩,🇸🇷,🇸🇿,🇸🇪,🇨🇭,🇸🇾,🇹🇼,🇹🇯,🇹🇿,🇹🇭,🇹🇱,🇹🇬,🇹🇰,🇹🇴,🇹🇹,🇹🇳,🇹🇷,🇹🇲,🇹🇨,🇹🇻,🇺🇬,🇺🇦,🇦🇪,🇬🇧,🇺🇸,🇻🇮,🇺🇾,🇺🇿,🇻🇺,🇻🇦,🇻🇪,🇻🇳,🇼🇫,🇪🇭,🇾🇪,🇿🇲,🇿🇼"
}
}
],
"CVViewFontList" : [
"AppleColorEmoji"
]
}

View File

@@ -1,13 +0,0 @@
set jsCode to "document.getElementById('output').value"
set json to missing value
tell application "Safari"
repeat
set json to (do JavaScript jsCode in current tab of window 1)
if (json is not missing value) then exit repeat
delay 0.5
end repeat
close current tab of window 1
end tell
return json

View File

@@ -1,81 +0,0 @@
<!DOCTYPE html>
<title>Emoji alias detection</title>
<style>
textarea {
font-family: monospace;
}
</style>
<p>Save the following as <tt>emoji.json</tt>:</p>
<textarea id="output" rows="50" cols="80"></textarea>
<script>
const VARIATION_SELECTOR_15 = String.fromCharCode(0xfe0e);
const VARIATION_SELECTOR_16 = String.fromCharCode(0xfe0f);
const EMOJI_SIZE = 32
function detectAliases(db) {
for (var i = 0; i < db.length; ++i) {
var emoji = db[i];
var raw = emoji.emoji;
if (!raw) {
continue;
}
if (raw.indexOf(VARIATION_SELECTOR_16) > -1) {
var candidates = [raw.replace(VARIATION_SELECTOR_16, ""), raw];
} else {
var candidates = [raw, raw + VARIATION_SELECTOR_16];
}
var aliases = candidates.filter(isColorEmoji);
emoji.emoji = aliases[0];
}
dump(db);
}
function isColorEmoji(candidate) {
// Draw the emoji twice using a different color each time. If the emoji
// draws as the same color regardless of what color we set, it's a color
// emoji.
return color(candidate, "#f00") === color(candidate, "#0f0");
}
var canvas = document.createElement("canvas");
canvas.width = canvas.height = EMOJI_SIZE;
function color(emoji, rgb) {
var context = canvas.getContext("2d");
context.clearRect(0, 0, canvas.width, canvas.height);
context.fillStyle = rgb;
context.textBaseline = "top";
context.font = EMOJI_SIZE+"px Arial";
context.fillText(emoji, 0, 0);
var data = context.getImageData(0, 0, EMOJI_SIZE, EMOJI_SIZE).data;
for (var i = 0; i < data.length; i += 4) {
if (data[i] === 0 && data[i + 1] === 0 && data[i + 2] === 0) {
continue;
}
return data[i].toString(16)
+ data[i + 1].toString(16)
+ data[i + 2].toString(16);
}
return "no colored pixel found";
}
function dump(db) {
var json = JSON.stringify(db, null, " ")
.replace(/^( +)(.+)\[\](,?)$/mg, "$1$2[\n$1]$3")
.replace(/,\n( *) /g, "\n$1, ");
document.getElementById("output").value = json + "\n";
}
var xhr = new XMLHttpRequest;
xhr.onload = function() {
detectAliases(JSON.parse(this.responseText));
};
xhr.open("GET", "emoji.json", false);
xhr.send(null);
</script>

View File

@@ -1,74 +1,69 @@
# frozen_string_literal: true
require 'emoji'
require 'json'
require_relative './emoji-test-parser'
names_list = File.expand_path('../NamesList.txt', __FILE__)
items = []
class UnicodeCharacter
attr_reader :code, :description, :aliases
_, categories = EmojiTestParser.parse(File.expand_path("../../vendor/unicode-emoji-test.txt", __FILE__))
seen_existing = {}
@index = {}
class << self
attr_reader :index
def fetch(code, *args, &block)
code = code.to_s(16).rjust(4, '0') if code.is_a?(Integer)
index.fetch(code, *args, &block)
for category in categories
for sub_category in category[:emoji]
for emoji_item in sub_category[:emoji]
unicodes = emoji_item[:sequences].sort_by(&:bytesize)
existing_emoji = nil
unicodes.detect do |raw|
existing_emoji = Emoji.find_by_unicode(raw)
end
existing_emoji = nil if seen_existing.key?(existing_emoji)
output_item = {
emoji: unicodes[0],
description: emoji_item[:description],
category: category[:name],
}
if existing_emoji
eu = existing_emoji.unicode_aliases
preferred_raw = eu.size == 2 && eu[0] == "#{eu[1]}\u{fe0f}" ? eu[1] : eu[0]
output_item.update(
emoji: preferred_raw,
aliases: existing_emoji.aliases,
tags: existing_emoji.tags,
unicode_version: existing_emoji.unicode_version,
ios_version: existing_emoji.ios_version,
)
seen_existing[existing_emoji] = true
else
output_item.update(
aliases: [emoji_item[:description].gsub(/\W+/, '_').downcase],
tags: [],
unicode_version: "12.0",
ios_version: "13.0",
)
end
output_item[:skin_tones] = true if emoji_item[:skin_tones]
items << output_item
end
end
def initialize(code, description)
@code = code.downcase
@description = description.downcase
@aliases = []
@references = []
self.class.index[@code] = self
end
def add_alias(string)
@aliases.concat string.split(/\s*,\s*/)
end
def add_reference(code)
@references << code.downcase
end
end
char = nil
missing_emoji = Emoji.all.reject { |e| e.custom? || seen_existing.key?(e) }
if missing_emoji.any?
$stderr.puts "Error: these `emoji.json` entries were not matched:"
$stderr.puts missing_emoji.map { |e| "%s (%s)" % [e.hex_inspect, e.name] }
exit 1
end
File.foreach(names_list) do |line|
case line
when /^[A-F0-9]{4,5}\t/
code, desc = line.chomp.split("\t", 2)
codepoint = code.hex
char = UnicodeCharacter.new(code, desc)
when /^\t= /
char.add_alias($')
when /^\tx .+ - ([A-F0-9]{4,5})\)$/
char.add_reference($1)
end
for emoji in Emoji.all.select(&:custom?)
items << {
aliases: emoji.aliases,
tags: emoji.tags,
}
end
trap(:PIPE) { abort }
items = []
variation_codepoint = Emoji::VARIATION_SELECTOR_16.codepoints[0]
for emoji in Emoji.all
item = {}
unless emoji.custom?
chars = emoji.raw.codepoints.map { |code| UnicodeCharacter.fetch(code) unless code == variation_codepoint }.compact
item[:emoji] = emoji.raw
item[:description] = chars.map(&:description).join(' + ')
end
item[:aliases] = emoji.aliases
item[:tags] = emoji.tags
items << item
end
puts JSON.pretty_generate(items)
.gsub("\n\n", "\n")
.gsub(/,\n( +)/) { "\n%s, " % $1[2..-1] }

121
db/emoji-test-parser.rb Normal file
View File

@@ -0,0 +1,121 @@
# frozen_string_literal: true
module EmojiTestParser
VARIATION_SELECTOR_16 = "\u{fe0f}"
SKIN_TONES = [
"\u{1F3FB}", # light skin tone
"\u{1F3FC}", # medium-light skin tone
"\u{1F3FD}", # medium skin tone
"\u{1F3FE}", # medium-dark skin tone
"\u{1F3FF}", # dark skin tone
]
HAIR_MODIFIERS = [
"\u{1F9B0}", # red-haired
"\u{1F9B1}", # curly-haired
"\u{1F9B2}", # bald
"\u{1F9B3}", # white-haired
]
module_function
def parse(filename)
File.open(filename, "r:UTF-8") do |file|
parse_file(file)
end
end
def parse_file(io)
data = []
emoji_map = {}
category = nil
sub_category = nil
io.each do |line|
begin
if line.start_with?("# group: ")
_, group_name = line.split(":", 2)
category = {
name: group_name.strip,
emoji: [],
}
data << category
sub_category = nil
elsif line.start_with?("# subgroup: ")
_, group_name = line.split(":", 2)
sub_category = {
name: group_name.strip,
emoji: [],
}
category[:emoji] << sub_category
elsif line.start_with?("#") || line.strip.empty?
next
else
row, desc = line.split("#", 2)
desc = desc.strip.split(" ", 2)[1]
codepoints, _ = row.split(";", 2)
emoji_raw = codepoints.strip.split.map { |c| c.hex }.pack("U*")
next if HAIR_MODIFIERS.include?(emoji_raw)
emoji_normalized = emoji_raw
.gsub(VARIATION_SELECTOR_16, "")
.gsub(/(#{SKIN_TONES.join("|")})/o, "")
emoji_item = emoji_map[emoji_normalized]
if SKIN_TONES.any? { |s| emoji_raw.include?(s) }
emoji_item[:skin_tones] = true if emoji_item
next
end
if emoji_item
emoji_item[:sequences] << emoji_raw
else
emoji_item = {
sequences: [emoji_raw],
description: desc,
}
emoji_map[emoji_normalized] = emoji_item
sub_category[:emoji] << emoji_item
end
end
rescue
warn "line: %p" % line
raise
end
end
[emoji_map, data]
end
end
if $0 == __FILE__
html_output = false
if ARGV[0] == "--html"
ARGV.shift
html_output = true
end
_, categories = EmojiTestParser.parse
trap(:PIPE) { abort }
if html_output
puts "<!doctype html>"
puts "<meta charset=utf-8>"
for category in categories
puts "<h2>#{category[:name]}</h2>"
for sub_category in category[:emoji]
puts "<h3>#{sub_category[:name]}</h3>"
puts "<ol>"
for char in sub_category[:emoji]
puts "<li>"
for sequence in char[:sequences]
codepoints = sequence.unpack("U*").map { |c| c.to_s(16).upcase }.join(" ")
printf '<span class=emoji title="%s">%s</span> ', codepoints, sequence
end
puts "#{char[:description]}</li>"
end
puts "</ol>"
end
end
else
require "json"
puts JSON.pretty_generate(categories)
end
end

View File

File diff suppressed because it is too large Load Diff

View File

@@ -66,7 +66,7 @@ li > span {
if (emoji.emoji) els[0].textContent = emoji.emoji
else {
var img = document.createElement('img')
img.src = "../images/emoji/" + emoji.aliases[0] + ".png"
img.src = "../images/" + emoji.aliases[0] + ".png"
els[0].appendChild(img)
}
els[1].textContent = emoji.description || ''

View File

@@ -1,9 +1,8 @@
Gem::Specification.new do |s|
s.name = "gemoji"
s.version = "3.0.0.rc1"
s.summary = "Emoji library"
s.description = "Character information and metadata for standard and custom emoji."
s.executables = ["gemoji"]
s.version = "4.0.0.pre0"
s.summary = "Unicode emoji library"
s.description = "Character information and metadata for Unicode emoji."
s.required_ruby_version = '> 1.9'
@@ -14,9 +13,7 @@ Gem::Specification.new do |s|
s.files = Dir[
"README.md",
"bin/gemoji",
"images/*.png",
"db/Category-Emoji.json",
"LICENSE",
"db/emoji.json",
"lib/**/*.rb",
]

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 898 B

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.9 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.7 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.2 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.0 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.1 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.4 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.0 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.0 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.1 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.2 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.5 KiB

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1004 B

View File

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.7 KiB

View File

@@ -1,4 +1,6 @@
# encoding: utf-8
# frozen_string_literal: true
require 'emoji/character'
require 'json'
@@ -9,14 +11,6 @@ module Emoji
File.expand_path('../../db/emoji.json', __FILE__)
end
def apple_palette_file
File.expand_path('../../db/Category-Emoji.json', __FILE__)
end
def images_path
File.expand_path("../../images", __FILE__)
end
def all
return @all if defined? @all
@all = []
@@ -24,17 +18,6 @@ module Emoji
@all
end
def apple_palette
return @apple_palette if defined? @apple_palette
data = File.open(apple_palette_file, 'r:UTF-8') { |f| JSON.parse(f.read) }
@apple_palette = data.fetch('EmojiDataArray').each_with_object({}) do |group, all|
title = group.fetch('CVDataTitle').split('-', 2)[1]
all[title] = group.fetch('CVCategoryData').fetch('Data').split(',').map do |raw|
TEXT_GLYPHS.include?(raw) ? raw + VARIATION_SELECTOR_16 : raw
end
end
end
# Public: Initialize an Emoji::Character instance and yield it to the block.
# The character is added to the `Emoji.all` set.
def create(name)
@@ -73,43 +56,144 @@ module Emoji
private
VARIATION_SELECTOR_16 = "\u{fe0f}".freeze
ZERO_WIDTH_JOINER = "\u{200d}".freeze
FEMALE_SYMBOL = "\u{2640}".freeze
MALE_SYMBOL = "\u{2642}".freeze
# Chars from Apple's palette which must have VARIATION_SELECTOR_16 to render:
TEXT_GLYPHS = ["🈷", "🈂", "🅰", "🅱", "🅾", "©", "®", "", ""].freeze
# Characters which must have VARIATION_SELECTOR_16 to render as color emoji:
TEXT_GLYPHS = [
"\u{1f237}", # Japanese “monthly amount” button
"\u{1f202}", # Japanese “service charge” button
"\u{1f170}", # A button (blood type)
"\u{1f171}", # B button (blood type)
"\u{1f17e}", # O button (blood type)
"\u{00a9}", # copyright
"\u{00ae}", # registered
"\u{2122}", # trade mark
"\u{3030}", # wavy dash
"\u{263a}", # smiling face
"\u{261D}", # index pointing up
"\u{270C}", # victory hand
"\u{270D}", # writing hand
"\u{2764}", # red heart
"\u{2763}", # heavy heart exclamation
"\u{2668}", # hot springs
"\u{2708}", # airplane
"\u{2600}", # sun
"\u{2601}", # cloud
"\u{2602}", # umbrella
"\u{2744}", # snowflake
"\u{2603}", # snowman
"\u{2660}", # spade suit
"\u{2665}", # heart suit
"\u{2666}", # diamond suit
"\u{2663}", # club suit
"\u{260e}", # telephone
"\u{2709}", # envelope
"\u{270F}", # pencil
"\u{2712}", # black nib
"\u{2702}", # scissors
"\u{26a0}", # warning
"\u{2B06}", # up arrow
"\u{2197}", # up-right arrow
"\u{27A1}", # right arrow
"\u{2198}", # down-right arrow
"\u{2B07}", # down arrow
"\u{2199}", # down-left arrow
"\u{2B05}", # left arrow
"\u{2196}", # up-left arrow
"\u{2195}", # up-down arrow
"\u{2194}", # left-right arrow
"\u{21A9}", # right arrow curving left
"\u{21AA}", # left arrow curving right
"\u{2934}", # right arrow curving up
"\u{2935}", # right arrow curving down
"\u{2721}", # star of David
"\u{262F}", # yin yang
"\u{271D}", # latin cross
"\u{25B6}", # play button
"\u{25C0}", # reverse button
"\u{23CF}", # eject button
"\u{2640}", # female sign
"\u{2642}", # male sign
"\u{267B}", # recycling symbol
"\u{2611}", # ballot box with check
"\u{2714}", # heavy check mark
"\u{2716}", # heavy multiplication x
"\u{303D}", # part alternation mark
"\u{2733}", # eight-spoked asterisk
"\u{2734}", # eight-pointed star
"\u{2747}", # sparkle
"\u{203C}", # double exclamation mark
"\u{2049}", # exclamation question mark
"\u{23}\u{20E3}", # keycap: #
"\u{2A}\u{20E3}", # keycap: *
"\u{30}\u{20E3}", # keycap: 0
"\u{31}\u{20E3}", # keycap: 1
"\u{32}\u{20E3}", # keycap: 2
"\u{33}\u{20E3}", # keycap: 3
"\u{34}\u{20E3}", # keycap: 4
"\u{35}\u{20E3}", # keycap: 5
"\u{36}\u{20E3}", # keycap: 6
"\u{37}\u{20E3}", # keycap: 7
"\u{38}\u{20E3}", # keycap: 8
"\u{39}\u{20E3}", # keycap: 9
"\u{2139}", # information
"\u{24C2}", # circled M
"\u{1F17F}", # P button
"\u{3297}", # Japanese “congratulations” button
"\u{3299}", # Japanese “secret” button
"\u{25AA}", # black small square
"\u{25AB}", # white small square
"\u{25FB}", # white medium square
"\u{25FC}", # black medium square
].freeze
private_constant :VARIATION_SELECTOR_16, :TEXT_GLYPHS
def parse_data_file
data = File.open(data_file, 'r:UTF-8') { |file| JSON.parse(file.read) }
data.each do |raw_emoji|
self.create(nil) do |emoji|
raw_emoji.fetch('aliases').each { |name| emoji.add_alias(name) }
if raw = raw_emoji['emoji']
unicodes = [raw, raw.sub(VARIATION_SELECTOR_16, '') + VARIATION_SELECTOR_16].uniq
unicodes.each { |uni| emoji.add_unicode_alias(uni) }
end
raw_emoji.fetch('tags').each { |tag| emoji.add_tag(tag) }
data = File.open(data_file, 'r:UTF-8') do |file|
JSON.parse(file.read, symbolize_names: true)
end
emoji.category = raw_emoji['category']
emoji.description = raw_emoji['description']
emoji.unicode_version = raw_emoji['unicode_version']
emoji.ios_version = raw_emoji['ios_version']
if "".respond_to?(:-@)
# Ruby >= 2.3 this is equivalent to .freeze
# Ruby >= 2.5 this will freeze and dedup
dedup = lambda { |str| -str }
else
dedup = lambda { |str| str.freeze }
end
append_unicode = lambda do |emoji, raw|
unless TEXT_GLYPHS.include?(raw) || emoji.unicode_aliases.include?(raw)
emoji.add_unicode_alias(dedup.call(raw))
end
end
# Add an explicit gendered variant to emoji that historically imply a gender
data.each do |raw_emoji|
raw = raw_emoji['emoji']
next unless raw
no_gender = raw.sub(/(#{VARIATION_SELECTOR_16})?#{ZERO_WIDTH_JOINER}(#{FEMALE_SYMBOL}|#{MALE_SYMBOL})/, '')
next unless $2
edit_emoji(find_by_unicode(no_gender)) do |emoji|
emoji.add_unicode_alias(
$2 == FEMALE_SYMBOL ?
raw.sub(FEMALE_SYMBOL, MALE_SYMBOL) :
raw.sub(MALE_SYMBOL, FEMALE_SYMBOL)
)
self.create(nil) do |emoji|
raw_emoji.fetch(:aliases).each { |name| emoji.add_alias(dedup.call(name)) }
if raw = raw_emoji[:emoji]
append_unicode.call(emoji, raw)
start_pos = 0
while found_index = raw.index(VARIATION_SELECTOR_16, start_pos)
# register every variant where one VARIATION_SELECTOR_16 is removed
raw_alternate = raw.dup
raw_alternate[found_index] = ""
append_unicode.call(emoji, raw_alternate)
start_pos = found_index + 1
end
if start_pos > 0
# register a variant with all VARIATION_SELECTOR_16 removed
append_unicode.call(emoji, raw.gsub(VARIATION_SELECTOR_16, ""))
else
# register a variant where VARIATION_SELECTOR_16 is added
append_unicode.call(emoji, "#{raw}#{VARIATION_SELECTOR_16}")
end
end
raw_emoji.fetch(:tags).each { |tag| emoji.add_tag(dedup.call(tag)) }
emoji.category = dedup.call(raw_emoji[:category])
emoji.description = dedup.call(raw_emoji[:description])
emoji.unicode_version = dedup.call(raw_emoji[:unicode_version])
emoji.ios_version = dedup.call(raw_emoji[:ios_version])
end
end
end
@@ -124,3 +208,6 @@ module Emoji
@unicodes_index
end
end
# Preload emoji into memory
Emoji.all

View File

@@ -1,3 +1,5 @@
# frozen_string_literal: true
module Emoji
class Character
# Inspect individual Unicode characters in a string by dumping its

View File

@@ -1,47 +0,0 @@
require 'emoji/extractor'
require 'fileutils'
module Emoji
module CLI
extend self
def dispatch(argv)
cmd = argv[0]
argv = argv[1..-1]
case cmd
when "extract"
public_send(cmd, argv)
when "help", "--help", "-h"
help
else
raise ArgumentError
end
return 0
rescue ArgumentError
$stderr.puts usage_text
return 1
end
def help
puts usage_text
end
def extract(argv)
path = argv.shift
raise ArgumentError if path.to_s.empty?
Emoji::Extractor.new(64, path).extract!
Dir["#{Emoji.images_path}/*.png"].each do |png|
FileUtils.cp(png, File.join(path, File.basename(png)))
end
end
def usage_text
<<EOF
Usage: gemoji extract <path>
EOF
end
end
end

View File

@@ -1,190 +0,0 @@
require 'emoji'
require 'fileutils'
module Emoji
class Extractor
EMOJI_TTF = "/System/Library/Fonts/Apple Color Emoji.ttc"
attr_reader :size, :images_path
def initialize(size, images_path)
@size = size
@images_path = images_path
end
def each(&block)
return to_enum(__method__) unless block_given?
File.open(EMOJI_TTF, 'rb') do |file|
font_offsets = parse_ttc(file)
file.pos = font_offsets[0]
tables = parse_tables(file)
glyph_index = extract_glyph_index(file, tables)
each_glyph_bitmap(file, tables, glyph_index, &block)
end
end
def extract!
each do |glyph_name, type, binread|
if emoji = glyph_name_to_emoji(glyph_name)
image_filename = "#{images_path}/#{emoji.image_filename}"
FileUtils.mkdir_p(File.dirname(image_filename))
File.open(image_filename, 'wb') { |f| f.write binread.call }
end
end
end
private
GENDER_MAP = {
"M" => "\u{2642}",
"W" => "\u{2640}",
}
FAMILY_MAP = {
"B" => "\u{1f466}",
"G" => "\u{1f467}",
"M" => "\u{1f468}",
"W" => "\u{1f469}",
}.freeze
FAMILY = "1F46A"
COUPLE = "1F491"
KISS = "1F48F"
def glyph_name_to_emoji(glyph_name)
return if glyph_name =~ /\.[1-5]($|\.)/
zwj = Emoji::ZERO_WIDTH_JOINER
v16 = Emoji::VARIATION_SELECTOR_16
if glyph_name =~ /^u(#{FAMILY}|#{COUPLE}|#{KISS})\.([#{FAMILY_MAP.keys.join('')}]+)$/
if $1 == FAMILY ? $2 == "MWB" : $2 == "WM"
raw = [$1.hex].pack('U')
else
if $1 == COUPLE
middle = "#{zwj}\u{2764}#{v16}#{zwj}" # heavy black heart
elsif $1 == KISS
middle = "#{zwj}\u{2764}#{v16}#{zwj}\u{1F48B}#{zwj}" # heart + kiss mark
else
middle = zwj
end
raw = $2.split('').map { |c| FAMILY_MAP.fetch(c) }.join(middle)
end
candidates = [raw]
else
raw = glyph_name.gsub(/(^|_)u([0-9A-F]+)/) { ($1.empty?? $1 : zwj) + [$2.hex].pack('U') }
raw.sub!(/\.0\b/, '')
raw.sub!(/\.(#{GENDER_MAP.keys.join('|')})$/) { v16 + zwj + GENDER_MAP.fetch($1) }
candidates = [raw]
candidates << raw.sub(v16, '') if raw.include?(v16)
candidates << raw.gsub(zwj, '') if raw.include?(zwj)
candidates.dup.each { |c| candidates << (c + v16) }
end
candidates.map { |c| Emoji.find_by_unicode(c) }.compact.first
end
# https://www.microsoft.com/typography/otspec/otff.htm
def parse_ttc(io)
header_name = io.read(4).unpack('a*')[0]
raise unless "ttcf" == header_name
header_version, num_fonts = io.read(4*2).unpack('l>N')
# parse_version(header_version) #=> 2.0
io.read(4 * num_fonts).unpack('N*')
end
def parse_tables(io)
sfnt_version, num_tables = io.read(4 + 2*4).unpack('Nn')
# sfnt_version #=> 0x00010000
num_tables.times.each_with_object({}) do |_, tables|
tag, checksum, offset, length = io.read(4 + 4*3).unpack('a4N*')
tables[tag] = {
checksum: checksum,
offset: offset,
length: length,
}
end
end
GlyphIndex = Struct.new(:length, :name_index, :names) do
def name_for(glyph_id)
index = name_index[glyph_id]
names[index - 257]
end
def each(&block)
length.times(&block)
end
def each_with_name
each do |glyph_id|
yield glyph_id, name_for(glyph_id)
end
end
end
def extract_glyph_index(io, tables)
postscript_table = tables.fetch('post')
io.pos = postscript_table[:offset]
end_pos = io.pos + postscript_table[:length]
parse_version(io.read(32).unpack('l>')[0]) #=> 2.0
num_glyphs = io.read(2).unpack('n')[0]
glyph_name_index = io.read(2*num_glyphs).unpack('n*')
glyph_names = []
while io.pos < end_pos
length = io.read(1).unpack('C')[0]
glyph_names << io.read(length)
end
GlyphIndex.new(num_glyphs, glyph_name_index, glyph_names)
end
# https://developer.apple.com/fonts/TrueType-Reference-Manual/RM06/Chap6sbix.html
def each_glyph_bitmap(io, tables, glyph_index)
io.pos = sbix_offset = tables.fetch('sbix')[:offset]
strike = extract_sbix_strike(io, glyph_index.length, size)
glyph_index.each_with_name do |glyph_id, glyph_name|
glyph_offset = strike[:glyph_data_offset][glyph_id]
next_glyph_offset = strike[:glyph_data_offset][glyph_id + 1]
if glyph_offset && next_glyph_offset && glyph_offset < next_glyph_offset
io.pos = sbix_offset + strike[:offset] + glyph_offset
x, y, type = io.read(2*2 + 4).unpack('s2A4')
yield glyph_name, type, -> { io.read(next_glyph_offset - glyph_offset - 8) }
end
end
end
def extract_sbix_strike(io, num_glyphs, image_size)
sbix_offset = io.pos
version, flags, num_strikes = io.read(2*2 + 4).unpack('n2N')
strike_offsets = num_strikes.times.map { io.read(4).unpack('N')[0] }
strike_offsets.each do |strike_offset|
io.pos = sbix_offset + strike_offset
ppem, resolution = io.read(4*2).unpack('n2')
next unless ppem == size
data_offsets = io.read(4 * (num_glyphs+1)).unpack('N*')
return {
ppem: ppem,
resolution: resolution,
offset: strike_offset,
glyph_data_offset: data_offsets,
}
end
return nil
end
def parse_version(num)
major = num >> 16
minor = num & 0xFFFF
"#{major}.#{minor}"
end
end
end

View File

@@ -1 +1,3 @@
# frozen_string_literal: true
require 'emoji'

View File

@@ -1,33 +0,0 @@
#!/bin/bash
# Usage: script/regenerate
#
# Note: only usable on OS X
#
# Combines `rake db:dump` and `db/aliases.html` filter to regenerate the
# `db/emoji.json` file using only emoji that are guaranteed to not render as
# ordinary characters on OS X.
set -e
case "$1" in
-h | --help )
sed -ne '/^#/!q;s/.\{1,2\}//;1d;p' < "$0"
exit 0
;;
esac
if [ "$(uname -s)" != "Darwin" ]; then
echo "Error: this script must be run under Mac OS X." >&2
exit 1
fi
bundle exec rake db:dump > db/emoji2.json
mv db/emoji2.json db/emoji.json
open -g -a Safari db/aliases.html
osascript db/aliases.applescript | sed '/^$/d' > db/emoji2.json
mv db/emoji2.json db/emoji.json
if git diff --exit-code --stat -- db/emoji.json; then
echo "Done. The file \`db/emoji.json\` remains unchanged."
fi

View File

@@ -1,4 +1,5 @@
require 'test_helper'
require_relative '../db/emoji-test-parser'
class EmojiTest < TestCase
test "fetching all emoji" do
@@ -7,7 +8,7 @@ class EmojiTest < TestCase
end
test "unicodes set contains the unicodes" do
min_size = Emoji.all.reject(&:custom?).size
min_size = Emoji.all.size
count = Emoji.all.map(&:unicode_aliases).flatten.size
assert count > min_size, "there were too few unicode mappings: #{count}"
end
@@ -21,7 +22,8 @@ class EmojiTest < TestCase
end
test "finding emoji by unicode" do
assert_equal "\u{1f604}", Emoji.find_by_unicode("\u{1f604}").raw
emoji = Emoji.find_by_unicode("\u{1f604}") # grinning face with smiling eyes
assert_equal "\u{1f604}", emoji.raw
end
test "finding nonexistent emoji by unicode returns nil" do
@@ -29,23 +31,13 @@ class EmojiTest < TestCase
end
test "unicode_aliases" do
emoji = Emoji.find_by_unicode("\u{2728}")
assert_equal ["\u{2728}", "\u{2728}\u{fe0f}"], emoji.unicode_aliases
end
test "unicode_aliases includes alternate position of VARIATION_SELECTOR_16" do
emoji = Emoji.find_by_unicode("\u{0031}\u{fe0f}\u{20e3}")
assert_equal ["\u{0031}\u{fe0f}\u{20e3}", "\u{0031}\u{20e3}\u{fe0f}"], emoji.unicode_aliases
emoji = Emoji.find_by_unicode("\u{2728}") # sparkles
assert_equal ["2728", "2728-fe0f"], emoji.unicode_aliases.map { |u| Emoji::Character.hex_inspect(u) }
end
test "unicode_aliases doesn't necessarily include form without VARIATION_SELECTOR_16" do
emoji = Emoji.find_by_unicode("\u{00a9}\u{fe0f}")
assert_equal ["\u{00a9}\u{fe0f}"], emoji.unicode_aliases
end
test "unicode_aliases for emoji that have gender variant include form with explicit gender" do
male_spy = Emoji.find_by_unicode("\u{1f575}\u{fe0f}")
assert_equal male_spy, Emoji.find_by_unicode("\u{1f575}\u{fe0f}\u{200d}\u{2642}\u{fe0f}")
emoji = Emoji.find_by_unicode("\u{00a9}\u{fe0f}") # copyright symbol
assert_equal ["00a9-fe0f"], emoji.unicode_aliases.map { |u| Emoji::Character.hex_inspect(u) }
end
test "emojis have tags" do
@@ -55,49 +47,104 @@ class EmojiTest < TestCase
assert emoji.tags.include?('pleased')
end
GENDER_EXCEPTIONS = [
"man_with_gua_pi_mao",
"woman_with_headscarf",
"man_in_tuxedo",
"pregnant_woman",
"isle_of_man",
"blonde_woman",
/^couple(kiss)?_/,
/^family_/,
]
test "emojis have valid names" do
aliases = Emoji.all.flat_map(&:aliases)
gender_mismatch = []
to_another_gender = lambda do |name|
case name
when *GENDER_EXCEPTIONS then name
else
name.sub(/(?<=^|_)(?:wo)?man(?=_|$)/) do |match|
match == "woman" ? "man" : "woman"
end
end
end
invalid = []
alias_count = Hash.new(0)
aliases.each do |name|
alias_count[name] += 1
invalid << name if name !~ /\A[\w+-]+\Z/
another_gender = to_another_gender.call(name)
gender_mismatch << another_gender unless aliases.include?(another_gender)
end
duplicates = alias_count.select { |_, count| count > 1 }.keys
assert_equal [], invalid, "some emoji have invalid names"
assert_equal [], duplicates, "some emoji aliases have duplicates"
assert_equal [], gender_mismatch, "missing gender variants"
end
test "missing or incorrect unicodes" do
source_unicode_emoji = Emoji.apple_palette.values.flatten
missing = source_unicode_emoji - Emoji.all.flat_map(&:unicode_aliases)
emoji_map, _ = EmojiTestParser.parse(File.expand_path("../../vendor/unicode-emoji-test.txt", __FILE__))
source_unicode_emoji = emoji_map.values
text_glyphs = Emoji.const_get(:TEXT_GLYPHS)
missing = 0
message = "Missing or incorrect unicodes:\n"
missing.each do |raw|
message << "#{raw} (#{Emoji::Character.hex_inspect(raw)})"
codepoint = raw.codepoints[0]
if candidate = Emoji.all.detect { |e| !e.custom? && e.raw.codepoints[0] == codepoint }
message << " - might be #{candidate.raw} (#{candidate.hex_inspect}) named #{candidate.name}"
source_unicode_emoji.each do |emoji|
emoji[:sequences].each do |raw|
found = Emoji.find_by_unicode(raw)
if text_glyphs.include?(raw)
assert_nil found, Emoji::Character.hex_inspect(raw)
next
end
next if found
message << "%s (%s)" % [Emoji::Character.hex_inspect(raw), emoji[:description]]
if found = Emoji.find_by_unicode(raw.gsub("\u{fe0f}", ""))
message << " - could be %s (:%s:)" % [found.hex_inspect, found.name]
end
message << "\n"
missing += 1
end
message << "\n"
end
assert_equal 0, missing.size, message
assert_equal 0, missing, message
end
test "raw representation does not include VARIATION SELECTOR 16 unless necessary" do
emoji = Emoji.all.select do |emoji|
!emoji.custom? && emoji.raw.end_with?("\u{fe0f}") && emoji.unicode_aliases.size == 2
end
assert_equal [], emoji
end
test "emoji have category" do
missing = Emoji.all.select { |e| !e.custom? && e.category.to_s.empty? }
missing = Emoji.all.select { |e| e.category.to_s.empty? }
assert_equal [], missing.map(&:name), "some emoji don't have a category"
emoji = Emoji.find_by_alias('family_man_woman_girl')
assert_equal 'People', emoji.category
assert_equal 'People & Body', emoji.category
categories = Emoji.all.map(&:category).uniq.compact
assert_equal [
"Smileys & Emotion",
"People & Body",
"Animals & Nature",
"Food & Drink",
"Travel & Places",
"Activities",
"Objects",
"Symbols",
"Flags",
], categories
end
test "emoji have description" do
missing = Emoji.all.select { |e| !e.custom? && e.description.to_s.empty? }
missing = Emoji.all.select { |e| e.description.to_s.empty? }
assert_equal [], missing.map(&:name), "some emoji don't have a description"
emoji = Emoji.find_by_alias('family_man_woman_girl')
@@ -110,27 +157,16 @@ class EmojiTest < TestCase
end
test "emoji have iOS version" do
missing = Emoji.all.select { |e| !e.custom? && e.ios_version.to_s.empty? }
missing = Emoji.all.select { |e| e.ios_version.to_s.empty? }
assert_equal [], missing.map(&:name), "some emoji don't have an iOS version"
emoji = Emoji.find_by_alias('family_man_woman_girl')
assert_equal '8.3', emoji.ios_version
end
test "custom emojis" do
test "no custom emojis" do
custom = Emoji.all.select(&:custom?)
assert custom.size > 0
custom.each do |emoji|
assert_nil emoji.raw
assert_equal [], emoji.unicode_aliases
end
end
test "custom emoji names" do
custom_names = Emoji.all.select(&:custom?).map(&:name)
assert custom_names.include?("shipit")
assert !custom_names.include?("+1")
assert 0, custom.size
end
test "create" do

View File

@@ -1,58 +0,0 @@
require 'test_helper'
require 'json'
require 'digest/md5'
unless images_extracted = File.directory?(File.join(Emoji.images_path, 'unicode'))
warn "Warning: skipping image integrity tests. Run \`rake images/unicode' on macOS to enable."
end
class IntegrityTest < TestCase
test "images on disk correlate 1-1 with emojis" do
images_on_disk = Dir["#{Emoji.images_path}/**/*.png"].map {|f| f.sub(Emoji.images_path, '') }
expected_images = Emoji.all.map { |emoji| '/%s' % emoji.image_filename }
missing_images = expected_images - images_on_disk
assert_equal 0, missing_images.size, "these images are missing on disk:\n #{missing_images.join("\n ")}\n"
extra_images = images_on_disk - expected_images
assert_equal 0, extra_images.size, "these images don't match any emojis:\n #{extra_images.join("\n ")}\n"
end
test "images on disk have no duplicates" do
hashes = Hash.new { |h,k| h[k] = [] }
Emoji.all.each do |emoji|
checksum = Digest::MD5.file(File.join(Emoji.images_path, emoji.image_filename)).to_s
hashes[checksum] << emoji
end
hashes.each do |checksum, emojis|
# Apple uses the same image for "black_medium_square" and "black_large_square":
expected_length = ("black_medium_square" == emojis[0].name) ? 2 : 1
assert_equal expected_length, emojis.length,
"These images share the same checksum: " +
emojis.map(&:image_filename).join(', ')
end
end
test "images on disk are 64x64" do
mismatches = []
Dir["#{Emoji.images_path}/**/*.png"].each do |image_file|
width, height = png_dimensions(image_file)
unless width == 64 && height == 64
mismatches << "%s: %dx%d" % [
image_file.sub(Emoji.images_path, ''),
width,
height
]
end
end
assert_equal ["/shipit.png: 75x75"], mismatches
end
private
def png_dimensions(file)
png = File.open(file, "rb") { |f| f.read(1024) }
png.unpack("x16N2")
end
end if images_extracted

4119
vendor/unicode-emoji-test.txt vendored Normal file
View File

File diff suppressed because it is too large Load Diff