Forecasters predict that fully autonomous vehicles could be commercially available in the next few years; and within the next few decades (i.e. 2040) half of all vehicles sold and 40 percent of vehicle travel could be autonomous (Lit- man 2019). While racing to realize this vision, the automotive industry is investing heavily into machine learning and other relevant AI technologies. To meet the increasing data demands of ML algorithms, fleets of vehicles are now deployed in multiple cities around the world and collecting massive amounts of data. These vehicles are equipped with various types of heterogeneous sensors, including but not limited to video, LIDAR, and RADAR.
To manage these vast amounts of automotive sensor data, companies are beginning to experiment with the use of KGs. In other industries, and for many years, KGs have proven invaluable for helping to manage data stored within enterprise data lakes, which are growing rapidly in popularity. More specifically, KGs help to enable the principles of FAIR data i.e. findability, accessibility, interoperability, and re-use across an enterprise.
Current research into the topic of neuro-symbolic fusion1 (Nickel et al. 2015; Sheth et al. 2019), however, is beginning to point to new and exciting uses of KGs. Essentially, KGs are a key source of high-quality domain knowledge and KGE technology is now enabling ML algorithms to more directly access this knowledge. Recent studies have already shown that the use of KGEs leads to improved performance and predictive capabilities (Chen et al. 2017; Wang et al. 2019; Myklebust et al. 2019). For this reason, we believe that neuro-symbolic fusion through KGEs may provide valuable knowledge needed to improve scene understanding for autonomous driving.
In this paper, we share our experience with generating and evaluating KGEs for the AD domain. To the best of our knowledge, this is the first attempt of its kind in this domain. Our investigation begins with two popular benchmark datasets from Aptiv and Lyft. From each of these datasets we generate multiple KGs with varying degrees of informational detail. The generated KGs focus on representing the various scenes, or situations, that an autonomous vehicle encounters on the road. The purpose of creating KGs with varying degrees of detail is to enable an examination of the relationship between KG detail and the quality of derivative embeddings. Each KG is then translated into a set of KGEs, each derived from one-of three popular embedding algorithms; including TransE (Bordes et al. 2013), RESCAL (Nickel, Tresp, and Kriegel 2011), and HoLE (Nickel, Rosasco, and Poggio 2016). Finally, the quality of each KGE is systematically evaluated based on the framework proposed in (Alshargi et al. 2019).
Our analysis of evaluating the KGEs along four dimensions – i.e. quality metrics, KG informational detail, algorithms, and datasets – leads to some interesting findings. First, we show that KGE quality significantly improves as the informational detail of a KG increases. Second, focusing on the evaluation measures, we report that some of the metrics such as the coherence measure may not be suitable to evaluate KGEs in this domain. When considering the effectiveness of KGE algorithms, we identify that the semantic transitional distance-based TransE algorithm captures type and relational semantics better than algorithms from the class of semantic matching-based models. It is interesting to note that these findings are consistent across the evaluations on two datasets. Finally, we report preliminary observations on using KGEs for two use cases from the AD domain. Specifically, we demonstrate how the scene/sub-scene understanding was improved as KG informational detail was increased, and how KGEs can be used to compute scene similarity.
The two primary contributions of this paper include: (1) a demonstration of the process of creating and evaluating KGEs for AD data, and (2) an (early) examination of the relationship between KG detail and the quality of KGEs. In Section 2, we discuss the construction of KGs from the benchmark automotive driving datasets. The translation of KGs to KGEs is explained in Section 3, while Section 4 focuses on their evaluation. Details of the technology used, as well as related work, will be discussed in each individual section. An investigation on the usefulness of semantics in the AD domain is discussed in Section 5. Finally, in Section 6 we conclude with a summary of our overall results and directions for future research.
To evaluate the KGEs for the AD domain, several KGs were created based on two popular benchmark datasets; NuScenes from Aptiv (Caesar et al. 2019) and Lyft-Level5 from Lyft (Kesten et al. 2019). To annotate the data from the datasets, a scene ontology was used.
2.1 Composition of the Datasets
Both the NuScenes and Lyft datasets follow a similar structure. NuScenes, for example, is divided into a set of 20 second driving segments/scenes, with 40 samples/sub-scenes per segment (i.e. one sample/sub-scene every 0.5 seconds). Each 20 second segment is associated with a temporal interval and spatial area, while each sample is associated with the data collected at a specific temporal instant (i.e. timestamp) and spatial coordinates.
The NuScenes dataset contains 850 driving segments with 34,149 samples. Each object and event detected in a sample is associated with one of 23 categories2.
The Lyft dataset contains 180 driving segments with 22,680 samples. Lyft has only 9 object and event categories that are used for annotating samples.
2.2 Scene Ontology
A scene is described as an observable volume of time and space (Henson et al. 2019). In the AD domain, a scene depicts a situation encountered by a vehicle. A few examples may include a vehicle stopped at a traffic light, cruising on the highway, or crashing into another vehicle. The concept of scene acts as the polestar with which all information about the vehicle, and its situation, are integrated. More specifi-cally, a scene may include information about time and location, the occurring events, and the participating objects.
A scene may also include sub-scenes. For example, consider a vehicle driving for 20 seconds on a highway. This drive may be represented as a single scene. However, during this drive the vehicle may encounter several different situations, each of which may also be represented as a scene.
Formally, Figure 1 shows the properties associated with a scene (depicted in Protege3).
Figure 1: Formal definition of a scene, as defined by the Scene Ontology
A subset of the events and features-of-interest (i.e. objects) represented within the Scene Ontology is shown in Figure 2. For this work, the Scene Ontology has been extended to subsume all concepts found in both the NuScenes and Lyft datasets.
2.3 Informational Detail of KGs
For each dataset, three distinct KGs are generated with differing levels of informational detail. It should be noted that the level of informational detail for each KG refers to the inclusion of additional information about scenes. The three levels include: (1) a base KG, (2) a KG with inferred type relations for objects/events, and (3) a KG with additional includes relations between scenes and object/events. It should be noted that this additional information does not correspond to an increase in the logical expressivity of the KGs. It is also worth mentioning that each KG includes the Scene Ontology along with the facts derived from each dataset.
Base KG Within the Base KG, each 20 second segment is instantiated as a scene, and each sample is instantiated as a sub-scene. The scene representing a 20 second segment is associated with a temporal interval (of 20 seconds) and
Figure 2: Subset of events and features-of-interest contained in the Scene Ontology.
Figure 3: Example KGs: (a) Base KG, (b) KG w/ inferred types, and (c) KG w/ include path
a spatial location (e.g. a city). This scene is also associated with a set of sub-scenes, representing the samples. Each sub-scene is associated with a temporal instant, spatial point, and the objects and events that participate in the scene. See Figure 3(a) for an example.
KG with Inferred Types In the Base KG, objects and events are explicitly typed to the most specific class possible. For example, an object instance representing a car is typed to the Car class. Because the Car class is a sub-class of Vehicle, then the instance is also a type of Vehicle. However, this knowledge is only implicit in the KG; implied by the semantics of the owl:subClassOf relation. To make this knowledge explicit, a reasoner is used to infer all implied types for each object and event instance. See Figure 3(b) for an example.
KG with Include Paths Within the Base KG, objects and events are associated with sub-scenes derived from samples of a 20 second drive. The scene representing the entire drive is associated with these participating objects and events through a two-hop path.
In order to make a more direct association between a scene representing a drive and the detected objects and events, new includes relations are added to the KG. See Figure 3(c) for an example.
Table 1 summarizes the statistics of the three KG versions we have generated.
Table 1: Statistics of the three KG versions generated from the NuScenes and Lyft datasets
The goal of learning embeddings from a KG is to represent the entities and relations in low-dimensional vector space while also maintaining the semantics contained in the KG. This transformation allows KGs to be more easily manipulated and used for downstream learning tasks (e.g. link prediction (Xiao, Huang, and Zhu 2016) and KG completion (Lin et al. 2015)). Vector representation of KGEs also allows background knowledge contained in KGs to be easily integrated with other input features of a machine learning model.
To select candidate algorithms for our experiments, we referred to the classification of KGE algorithms established by (Wang et al. 2017) and (Sharma, Talukdar, and others 2018). KGE algorithms are categorized into two main classes: (1) Transitional distance-based algorithms and (2) Semantic Matching Models. For transitional distance-based algorithms (i.e. additive methods) the scoring function is composed of distance measures, and vector addition/subtraction is used to capture the vector interaction. For Semantic Matching Models (i.e. multiplicative methods) the scoring function is based on a similarity measure, and entity-relation-entity interaction is captured via a multiplicative score function. We initially selected one algorithm from each class. Specifically, we selected TransE from the former category and RESCAL from the latter category. RESCAL, however, has limitations in handling big KGs due to its high space and time complexity. As a result, we also included HolE into our experiments, which is a more space and memory-efficient successor of RESCAL.
3.1 Preliminaries
Next, the symbols and notations used throughout the paper are introduced, along with important details of the KGE algorithms.
Notation: Given a set of entities E and set of relations R, we define KG to be a set of triples () where L is the set of literals. We consider
where C is set of concepts from the ontology and N is set of individuals. Lowercase bold characters represent vectors of an entity or relation and uppercase bold characters represent a set of vectors. For example,
is an embedding vector of
. Most of the embedding algorithms – including some used in our evaluation – generate vectors of dimension d to represent entities
for
and
to represent relations
. However, some algorithms learn a projection matrix
to represent relations. The scoring function
used in each algorithm is different. Transitional distance-based models use distance measures whereas semantic matching based methods use similarity measures. The learning of embeddings involve optimizing parameter
in the loss function
) where T is the set of positive triples and
is the set of corrupted triples. When considering time and space complexities of each algorithm, we consider n = |E|, m = |R| and
to be the number of training triples.
The details of each algorithm used are briefly discussed below;
TransE TransE is considered the most representative of the translational distance-based class of algorithms. Given a triple (h, r, t), TransE represents r as a translation vector from h to t. Hence when the triple (h, r, t) holds true. The scoring function of TransE
is defined as the negative distance between h + r and t, and when the (h, r, t)
holds, is expected to be large.
TransE is one of the most efficient KGE algorithms having O(nd + md) space complexity and time complexity. Despite it’s benefits, TransE falls short in capturing 1-N, N-1 and N-N relations in KGs.
RESCAL RESCAL belongs to the semantic matching/multiplicative class of KGE algorithms. RESCAL is an expressive model which takes into account the inherent structure in multi-relational KGs and captures complex patterns over multiple hops in the KG. It represents each relation r as matrix that captures all the interaction between vectors (h, t) of the entities h and r. The scoring function
is a bi-linear function which computes pairwise interaction of entities with respect to each relation r.
The main limitation of using RESCAL with big KGs is due to its high space and time complexity. RESCAL has space complexity and
time complexity.
HolE HolE is an efficient successor of RESCAL and addresses the high space complexity of RESCAL while retaining the expressive power. HolE represents both entities and relations as vectors in . Given a triple (h, r, t), HolE first creates a compositional vector using circular correlation operation which aims at compressing the pairwise interaction;
The total score for a given fact is then calculated by using the function that considers both the compositional vector and the relation vector.
Due to the use of circular correlation, HolE achieves O(nd + md) space complexity and time complexity.
3.2 Visualizing KG Embeddings
We selected 10 driving segments (including their samples) from each dataset - NuScenes and Lyft - and created “mini” KGs to visualize the embeddings in 2-dimensional (2D) space. After experimenting with both PCA (Wold, Esbensen, and Geladi 1987) and t-Distributed Stochastic Neighbor Embedding (t-SNE)(Maaten and Hinton 2008), t-SNE was selected for dimensionality reduction as its 2D projections yielded more meaningful clusters than PCA for the generated embeddings. The embedding dimension d was set to 100 when generating embeddings for all our experiments.
Our extended Scene Ontology identifies events and features-of-interests (FoI) as top-level classes in the ontology, and each instance of event or FoI are linked to Scenes via the includes relation. FoIs are related to events through the isParticipantOf relation. Therefore, we first look at how FoIs and events are manifested in the embedding space for each dataset.
NuScenes KG Embeddings Figure 4 shows how the events and FoIs form clusters in the embedding space based on their type. For the sake of brevity, only cars and the events in which they participate are highlighted. From this visualization, you can see that instances of events such as stopped car, moving car and parked car are clustered around the instances of car. The embeddings represented in this fig-ure are generated from TransE on the “Base KG”.
Figure 4: Clustering of FoIs with events in NuScenes dataset
Lyft KG Embeddings Figure 5 shows a similar visualization of the embeddings from the Lyft “Base KG”. This image shows how the events in Lyft are clustered together with FoIs. It may be noticed that Lyft contains fewer clusters than NuScenes. This is the case since the Lyft dataset only contains annotations for a few FoIs. Similar to what we’ve seen with NuScenes, Lyft embeddings also show how the instances of events such as stopped car, parked car and driving straightforward are clustered around instances of car.
The primary objective of our evaluation is to determine how well the salient features and rich semantics of KGs, such as type and relation semantics, transfer to learned embeddings. Our evaluation deviates from most prior work evaluating KGE algorithms, which focus on evaluating the performance of some extrinsic downstream task (such as entity classification). Here, we focus more on an intrinsic evaluation of embeddings.
Figure 5: Clustering of FoIs with events in Lyft dataset
4.1 Evaluation of KG Embeddings
There exists a large body of literature that evaluates the effect of using KGEs on a downstream task. To our knowledge, however, there’s only one recent work which introduces metrics to evaluate and quantify the intrinsic quality of embeddings. Of the metrics introduced in (Alshargi et al. 2019), we adapt three metrics for our evaluation: categorization measure, coherence measure, and semantic transition distance. Figure 6 depicts the four dimensions involved in our evaluation – i.e. quality metrics, datasets, KGE algorithms and KGs with varying degrees of informational detail.
Figure 6: Four dimensions involved in our evaluation
Next we provide a brief overview of each of the evaluation metrics;
The categorization measure captures how well the entities that are “typed” by the same background concept cluster together. For example, all the entities that are typed by the concept Car () share common characteristics and should be clustered together. Hence this metric is computed by taking the cosine similarity
of the averaged embedding vector (equation 5) of all such entities and the embedding vector of the background concept k,
.
(6) The coherence measure captures whether the adjacent entities in the embedding space share a common background concept. In introducing this measure, authors hypothesise that in the ideal case, all the entities that are typed by the same background concept should form a cluster and the background concept should be the centroid of this cluster. This is quantified (equation 7) by taking n closest entities of the background concept
and taking the proportion of which that are actually typed by
. For our experiments, we choose n to be 1000.
The semantic transition distance is a widely used metric in word embedding literature and re-introduced to KGEs to capture the relational semantics of KGs. For example, assume is asserted as the domain of the property
and
is asserted as its range. Then, if (
) is correctly represented in the embedding space, the transition distance between
should be close to
. This is formally represented in equation 8 where
is semantic transition distance of relation r and s denotes cosine similarity.
(8) Next we report our evaluation results of these three metrics with respect to each dataset/algorithm.
Evaluation on the Lyft Dataset Figure 7 summarizes the results of the categorization measure computed on the embeddings generated from three algorithms on three KG versions. It is clear from the figures that TransE performs better compared to RESCAL and HolE and the categorization quality is mostly better in the KG with more informational detail (i.e. KG w/ include paths) compared to other two less expressive variants. In our experimental setting, this measure is computed considering the top level concepts (FoIs and events) in the Scene Ontology.
Figure 7: Categorization measure computed for different KGs on the embeddings generated from (a) TransE, (b) RESCAL and (c) HolE
The results of the coherence measure, as depicted in fig-ure 8, shows a similar trend as the categorization measure; TransE is performing better and both RESCAL and HolE fail to generate entity clusters with high purity and closer to the background concept of those entities. It is interesting to note that, the KG with highest informational detail shows significant improvement in coherence measure on the embeddings generated from TransE.
Figure 8: Coherence measure computed for different KGs on the embeddings generated from (a) TransE, (b) RESCAL and (c) HolE
Next we look at how the relational semantics in KGs are transferred to KGEs by computing semantic transition distance for 11 relations defined in the Scene ontology. As per figure 9, KGs with include paths are able to capture relational semantics better than the other two variants across all three algorithms. An interesting observation to note here is that the isPartOf relation performs significantly better in KGs with include paths across all three algorithms even though we have only added implicit include relations. A possible explanation could be that the implicit include paths make the relationship between scenes and sub-scenes stronger in KGs with the highest informational detail.
Figure 9: Semantic transition distance computed for differ- ent KGs on the embeddings generated from (a) TransE, (b) RESCAL and (c) HolE
Evaluation on the NuScenes Dataset The evaluation process on the NuScenes dataset is similar to Lyft. However, we report that RESCAL was not scalable to the NuScenes KGs (having 10.8+ million triples and 2.1+ million entities). Therefore, we evaluate NuScenes KGEs only on TransE and HolE.
The results of the categorization measure for the NuScenes dataset follows a similar trend as Lyft (see Figure 10). TransE embeddings on KG w/ include paths yields the best categorization performance, with the exception of a few concepts where HolE outperforms on the base KG. Except for these few outliers, the results show that higher level of informational detail in KG achieves better categorization irrespective of the KGE algorithms used for training.
Figure 10: Categorization measure computed for different KGs on the embeddings generated from (a) TransE and (b) HolE
The benefits of information detail in KG are portrayed well in the results of coherence measure (see Figure 11). Even though the coherence measure, for many concepts, are either non-existent or closer to zero, the KG w/ include paths significantly outperform the other two KG variants for those values that exist.
Figure 11: Coherence measure computed for different KGs on the embeddings generated from (a) TransE and (b) HolE
The results of the semantic transition distance for TransE show consistent patterns similar to Lyft (see Figure 12). HolE, however, shows that base KGEs perform on par with embeddings trained on the KG w/ include paths.
Figure 12: Semantic transition distance computed for differ- ent KGs on the embeddings generated from (a) TransE and (b) HolE
4.2 Discussion
The evaluation of KGEs for AD domain lead to some interesting observations. We discuss our observations in three perspectives: (1) KGE algorithmic perspective, (2) evaluation measures, and (3) various levels of KG informational detail.
First, looking at the overall performance of KGE algorithms, TransE performs better than RESCAL and HolE in capturing both type and relational semantics. TransE is also scalable to large KGs and shows consistent performance across datasets. In addition to RESCAL’s space and time complexity, it’s performance on all three metrics is worse than TransE and HolE. Even though HolE’s performance is sub-optimal compared to TransE, it was consistent across the two datasets and the derived KGs. We hypothesize that the better performance of TransE on all three metrics is due to the way it is learning embeddings; i.e. using the translational distance based scoring function inspired by word embedding algorithms. The KGE quality metrics introduced by (Alshargi et al. 2019) are inspired by the word embedding literature. They evaluated these metrics on KGEs generated from word embedding based RDF2Vec (Ristoski and Paul- heim 2016) algorithm. Hence, it may be worth examining whether these metrics are suitable only for evaluating embeddings generated from translational distance / word embedding inspired KGE algorithms.
Second, when considering the evaluation measures used, we observe that the coherence measure is not very meaningful in this domain to evaluate the quality of KGEs. The entities are mostly clustered based on either scenes/sub-scenes or FoIs/events. Hence, the n most similar entities to a class (e.g. Human) are mostly not homogeneous, resulting in zero or close to zero coherence value.
Third, it has been consistently shown across multiple datasets and algorithms that KGs with the highest levels of informational detail are able to capture both type and relational semantics better than the other two less expressive variants. This discovery leads to an interesting future direction for research. To the best of our knowledge, all existing KGE algorithms in the literature are evaluated on base KGs (i.e. KGs without any inference). Therefore, it stands to reason that a KG embedding derived from a KG with more informational detail should capture more salient features and rich semantics of the KG. Such informational details can be captured either through pre-processing or by automatically extracted by the KGE algorithm.
In the previous section we looked at different quality aspects of embeddings which are common to KGs in any domain. Here, we report additional experiments which look at certain aspects specific to AD domain and scene understanding.
5.1 Scene/sub-scene Relationship
An understanding of complex AD scenes is an important task in AD domain. The ability to distinguish one complex scene from another requires looking at (1) how the scene/sub-scene relationship (formally defined by isPartOf relation) is captured by the embeddings in different KG versions, and (2) how well the FoIs and events cluster based on the scenes/subscenes they belong to.
Figure 13 shows the manifestation of the scene-subscene relationship that is moving from a “no-relation” (figure 13(a)) in the base KG to a more “meaningful” one (figure 13(c)) in the KG with the highest informational detail. Then we look at how the various levels of informational detail in KGs affect the clustering of FoIs and events of a scene based on scene/sub-scene relationship.
Figure 13: Clustering of scenes with sub-scenes in different KG versions of Lyft: (a) base KG, (b) KG w/ inferred types, and (c) KG w/ include paths
Figure 14 shows how the FoI/event dominant clusters in the base KG transfer to a clustering based on 10 scenes in the KG w/ include paths. Interestingly, we can still see small clusters formed based on FoIs/events inside the larger scene clusters. This suggests that the KGs with access to more informational detail are able to distinguish a scene by both participating FoIs/events as well as scene/sub-scene relationships.
Figure 14: Clustering of FoIs and events together with scenes/sub-scenes in different KG versions: (a) base KG, (b) KG w/ inferred types, and (c) KG w/ include paths
5.2 KGEs for Computing Scene Similarity
We report preliminary results of using KGEs for computing scene similarity. Our objective here is to determine whether two scenes are similar by considering only KGEs. Given a set of scene pairs, we calculate the cosine similarity between the KGE vectors of two scenes in a pair and then select pairs with the highest cosine similarity. Figure 15(a) shows the two most similar sub-scenes when pairs include sub-scenes from the same parent scene. With further investigation, we found that these two sub-scenes are in fact subsequent frames (or samples) from the same 20 second driving segment. Figure 15(b) shows the two most similar sub-scenes when pairs contain only sub-scenes from different scenes. It is interesting to note that the KGE based similarity computation was able to identify two sub-scenes which are not visually similar, but share common characteristics. For example, the black string of objects in Figure 15(b) (Left) are barriers (a Static Object) and the orange string of objects in Figure 15(b) (Right) are set of stopped cars.
Figure 15: Most similar sub-scenes computed using KGEs trained on NuScenes Base KG; (a) sub-scenes from the same scene and (b) sub-scenes from two different scenes
In this paper, we present an evaluation of KGEs for the autonomous driving domain that considers multiple datasets, metrics, algorithms and levels of informational detail. This evaluation supports the hypothesis that a KG with more detailed information yields higher quality KG embeddings with respect to both type and relational semantics. Furthermore, this evaluation highlights an important question about the suitability of metrics used in the existing literature, to evaluate a wide range of KGE algorithms. Finally, opening rich areas for future research, we present an early investigation into the use of KGEs for two important use-cases from the AD domain: scene/sub-scene understanding and computing scene similarity.
[Alshargi et al. 2019] Alshargi, F.; Shekarpoour, S.; Soru, T.; and Sheth, A. 2019. Metrics for evaluating quality of embeddings for ontological concepts. In AAAI 2019 Spring Symposium on Combining Machine Learning with Knowledge Engineering (AAAI-MAKE 2019).
[Bordes et al. 2013] Bordes, A.; Usunier, N.; Garcia-Duran, A.; Weston, J.; and Yakhnenko, O. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems, 2787–2795.
[Caesar et al. 2019] Caesar, H.; Bankiti, V.; Lang, A. H.; Vora, S.; Liong, V. E.; Xu, Q.; Krishnan, A.; Pan, Y.; Bal-
dan, G.; and Beijbom, O. 2019. nuscenes: A multimodal dataset for autonomous driving. ArXiv abs/1903.11027.
[Chen et al. 2017] Chen, M.; Tian, Y.; Yang, M.; and Zan- iolo, C. 2017. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, 1511–1517. AAAI Press.
[Henson et al. 2019] Henson, C.; Schmid, S.; Tran, T.; and Karatzoglou, A. 2019. Using a knowledge graph of scenes to enable search of autonomous driving data. In Proceedings of the 2019 International Semantic Web Conference (ISWC 2019).
[Kesten et al. 2019] Kesten, R.; Usman, M.; Houston, J.; Pandya, T.; Nadhamuni, K.; Ferreira, A.; Yuan, M.; Low, B.; Jain, A.; Ondruska, P.; Omari, S.; Shah, S.; Kulkarni, A.; Kazakova, A.; Tao, C.; Platinsky, L.; Jiang, W.; and Shet, V. 2019. Lyft level 5 av dataset 2019. https://level5.lyft.com/ dataset/.
[Lin et al. 2015] Lin, Y.; Liu, Z.; Sun, M.; Liu, Y.; and Zhu, X. 2015. Learning entity and relation embeddings for knowledge graph completion. In Twenty-ninth AAAI conference on Artificial Intelligence.
[Litman 2019] Litman, T. 2019. Autonomous vehicle imple- mentation predictions: Implications for transport planning. https://www.vtpi.org/avip.pdf.
[Maaten and Hinton 2008] Maaten, L. v. d., and Hinton, G. 2008. Visualizing data using t-sne. Journal of machine learning research 9(Nov):2579–2605.
[Myklebust et al. 2019] Myklebust, E. B.; Jimenez-Ruiz, E.; Chen, J.; Wolf, R.; and Tollefsen, K. E. 2019. Knowledge graph embedding for ecotoxicological effect prediction. In International Semantic Web Conference, 490–506. Springer.
[Nickel et al. 2015] Nickel, M.; Murphy, K.; Tresp, V.; and Gabrilovich, E. 2015. A review of relational machine learning for knowledge graphs. Proceedings of the IEEE 104(1):11–33.
[Nickel, Rosasco, and Poggio 2016] Nickel, M.; Rosasco, L.; and Poggio, T. 2016. Holographic embeddings of knowledge graphs. In Thirtieth AAAI Conference on Artificial Intelligence.
[Nickel, Tresp, and Kriegel 2011] Nickel, M.; Tresp, V.; and Kriegel, H.-P. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on International Conference on Machine Learning, 809–816. Omnipress.
[Ristoski and Paulheim 2016] Ristoski, P., and Paulheim, H. 2016. Rdf2vec: Rdf graph embeddings for data mining. In International Semantic Web Conference, 498–514. Springer.
[Sharma, Talukdar, and others 2018] Sharma, A.; Talukdar, P.; et al. 2018. Towards understanding the geometry of knowledge graph embeddings. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 122–131.
[Sheth et al. 2019] Sheth, A.; Gaur, M.; Kursuncu, U.; and Wickramarachchi, R. 2019. Shades of knowledge-infused
learning for enhancing deep learning. IEEE Internet Computing 23(6):54–63.
[Wang et al. 2017] Wang, Q.; Mao, Z.; Wang, B.; and Guo, L. 2017. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering 29(12):2724–2743.
[Wang et al. 2019] Wang, H.; Zhang, F.; Zhao, M.; Li, W.; Xie, X.; and Guo, M. 2019. Multi-task feature learning for knowledge graph enhanced recommendation. In The World Wide Web Conference, 2000–2010. ACM.
[Wold, Esbensen, and Geladi 1987] Wold, S.; Esbensen, K.; and Geladi, P. 1987. Principal component analysis. Chemometrics and intelligent laboratory systems 2(1-3):37–52.
[Xiao, Huang, and Zhu 2016] Xiao, H.; Huang, M.; and Zhu, X. 2016. From one point to a manifold: knowledge graph embedding for precise link prediction. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, 1315–1321. AAAI Press.
TransE RESCAL HolE
Table 2: Results of categorization measure computed on embeddings generated from TransE, RESCAL and HolE
Table 3: Results of coherence measure computed on embeddings generated from TransE, RESCAL and HolE
Table 4: Semantic transition distance computed on embeddings generated from TransE, RESCAL and HolE
TransE HolE
Table 5: Results of categorization measure computed on embeddings generated from TransE and HolE
TransE HolE
Table 6: Results of coherence measure computed on embeddings generated from TransE and HolE
Table 7: Semantic transition distance computed on embeddings generated from TransE and HolE