Biomass, which is typically defined as the over-dry mass of the above ground portion of a group of vegetation in forestry (Vazirabad & Karslioglu, 2011), is one of the most important parameters of roadside vegetation, such as grasses and trees. Automatic estimation of grass biomass can be useful in various real-world applications, including monitoring roadside grass growth conditions, enforcing effective roadside management, and evaluating road safety. One typical example regarding the use of biomass is to identify the level of fire risk due to the presence of high, dense and dry roadside grasses, which are often characterised by high biomass. From the perspective of the transport, roadside grasses of high biomass can potentially become a big fire threat to the safety of vehicles, particularly in remotely located rural regions. Enforcing regular and frequent checks on roadside grass conditions by humans in a large state road network is often a big burden for transport authorities in terms of labour, cost, and time investments. Thus, it is of great significance to develop systems that are capable of automatically estimating the biomass of roadside grasses and precisely identifying those roadside regions with high fire risk, whereby necessary actions can be carried out to prevent possible fire threats such as burning or cutting the grasses.
A typical method of calculating biomass is to conduct field surveys, which often include destructive plant sampling within a sampling region and calculating the weight after overdrying them (C. Royo & Villegas, 2011). It is one of the most accurate ways for obtaining biomass. Obviously, this method is heavily dependent on human efforts and requires extensive time, labour and cost, as well as expertise and equipment support. More importantly, it is unsuitable for automatic processing of data from large-scale fields.
The vast majority of existing solutions to automatically estimating the above-ground biomass of vegetation have been investigated using remote sensing methods (Lu, et al., 2016). The basic assumption of remote sensing methods for biomass estimation is that the mass of biomass is proportional to the volume of the vegetation and accordingly existing methods mainly base the biomass estimation on the upper layer of the canopy. The remotely sensed data can be captured using different types of sensors mounted on airborne, space-borne or terrestrial platforms. Optical spectral sensors are one of the most common ways of acquiring remotely sensed data with various spatial, spectral, radiometric and temporal resolutions. Typical examples of optical measurements are Vegetation Indices (VIs) (Schaefer & Lamb, 2016), spectral bands (Sibanda, Mutanga, & Rouget, 2016) and spatial image texture (Lu, et al., 2016). However, it is often difficult to obtain high quality optical data in frequent cloud conditions, and the optical measurements are prone to be affected by variations in solar radiation. Not all vegetation indices are closely related with biomass. The widely used Synthetic Aperture Radar (SAR) (Santi, et al., 2017) and LIght Detection And Ranging (LIDAR) (Lei Zhang & Grift, 2012), (Andújar, et al., 2016) sensors offer a better tolerance to weather and light conditions and are capable of collecting three dimensional distribution of structures within vegetation. Thus, they allow precise analysis on the characteristics of vegetation including biomass. However, these sensors are largely dependent on satellite or airborne platforms in existing studies, which leads to high costs and low flexibility. To provide more economical and convenient ways for data collection, more recent advances have tended to adopt drone-based sensors (Tang & Shao, 2015), (Kachamba, Ørka, Gobakken, Eid, & Mwase, 2016), (Fan, et al., 2017) or ground-based sensors such as ultrasonic sensor (Moeckel, Safari, Reddersen, Fricke, & Wachendorf, 2017), (Chang, et al., 2017) and mobile laser scanner (Ryding, Williams, Smith, & Eichhorn, 2015),(Li, Li, Zhu, & Li, 2016). Similar to satellite or airborne data, data collected using drone-mounted sensors reflect predominantly the upper canopy layers. Ground-based sensors can capture the whole above-ground structure of vegetation and thus they are suitable for biomass estimation in both large-scale and site-specific field surveys.
Except for remote sensing methods, another relatively less investigated method for biomass estimation is to use ground-based digital image or video data captured using ordinary cameras. Compared with remotely sensed data, ground-based images or video are relatively easier to collect using everyday devices such as ordinary cameras, smart phones and tablets, and can be operated by general people without requiring specialized knowledge. For the purpose of this paper, our industry partner - Department of Transport and Main Roads (DTMR), Queensland, Australia collects roadside video data from main state roads in Queensland using vehicle-mounted cameras, thereby human are employed to visually assess roadside conditions, such as vegetation species, height, fuel load, and potential safety threats to roads. For real-world applications where only ground-based video data are available, it is crucially important to develop automatic systems capable of estimating biomass from video frames.
Estimating biomass from ground-based digital image or video data is still a seldom investigated field. Studies (Juan & Xin-yuan, 2009), (Sritarapipat, Rakwatin, & Kasetkasem, 2014) exploited the way of estimating the height of vegetation from ground-based digital images. The height was calculated by measuring the distance between reference markers, which were pre-set manually on different parts of vegetation. Thus, these methods cannot be directly used for automatic applications. In our previous work (Verma, Zhang, & Stockwell, 2017), we have proposed the Vertical Orientation Connectivity of Grass Pixels (VOCGP) approach to automatically predict roadside grass biomass based on the grass height in images. The VOCGP approach segments brown grass pixels using an Artificial Neural Network (ANN) classifier with color and texture features, and detects the dominant texture direction at every pixel by performing Gabor-based votes on local texture. It then obtains the length of continuously connected grass pixels along every image column and takes an average length as the predicted biomass. However, the approach estimates the biomass predominantly based on the grass height and has largely ignored the contribution of the grass density to the biomass. The grass density is also an important for determining the grass biomass.
To solve the drawbacks in existing methods, this paper proposes Density Weighted Connectivity of Grass Pixels (DWCGP) to automatically estimate the biomass of roadside grasses in ground-based images. The DWCGP extends the VOCGP approach by jointly considering both grass height and density in the estimation of biomass, and thus it is expected that the DWCGP can produce more accurate estimation results. The main novelties in this paper are (a) a novel concept for determining the grass pixel orientation, height, and density without using any reference object; and (b) a novel integrated framework based on grass region segmentation and vertical grass orientation for grass biomass calculation. To the best of our knowledge, this is one of the first attempts that estimate grass biomass on ground-based data using image processing techniques.
The original contributions of this paper are as follows:
a) A concept of DWCGP for estimating grass biomass based on local texture features in a
sampling window is presented. The DWCGP measures both the grass height and density to quantify the fuel loads of grasses, leading to accurate prediction of the biomass.
b) An integrated framework for DWCGP calculation based on the results of grass vs. non-
grass pixel classification and vertical vs. non-vertical orientation detection is presented. Because the framework does not require manually setting up reference makers, nor the availability of specified equipment rather than a digital camera, it can be directly applied into site-specific analysis in a large-scale field.
c) An evaluation of DWCGP is presented by conducting a large number of experiments and
comparisons with ground truths of both objective biomass and subjective density of roadside grasses collected from field surveys. A comparative analysis to show the effectiveness of DWCGP in supporting fire-prone road identification is included.
The remainder of the paper is organized as follows. Section 2 discusses related work. Section 3 introduces the proposed DWCGP approach. Experimental results are presented in Section 4. Section 5 draws the conclusions.
This section reviews prior work on grass region segmentation and grass height estimation. Although intensive works (Hamuda, Glavin, & Jones, 2016) have been reported on vegetation or crop analysis and scene understanding, only few studies have specifically focused on roadside grass analysis. Compared with grassland vegetation, roadside grasses often have a more visible profile of the whole structure (e.g. appearance, geometry and length of grass stems), which is particularly important for analysing tall grasses. By contrast, analysis of grassland vegetation is often restricted to the upper layer of grasses only.
2.1 Grass Region Segmentation
Existing work relevant to grass region segmentation can be approximately divided into two groups, including visible and invisible feature approaches.
a) Visible feature approaches extract visual properties of vegetation such as shape, texture,
geometry, structure and color in the visible spectrum to distinguish them from other objects such as sky, road and soil. They can be further divided into three groups: (1) approaches that extract features from a Region Of Interest (ROI) for object classification. Campbell et al. (Campbell, Thomas, & Troscianko, 1997) adopted a self-organizing feature map for object segmentation using color and Gabor texture, and a multi-layer perceptron for classifying 11 outdoor objects. In (Haibing, Shirong, & Chaoliang, 2014), the mean shift was used to segment an image into local regions, and pixels in each region were mapped into the learnt Scale-Invariant Feature Transform (SIFT) words to obtain a histogram for recognizing five objects. In (Harbas & Subasic, 2014), the ROI in video was first estimated by optical flow, and color and Continuous Wavelet Transform (CWT) based texture were then extracted from the ROI for vegetation classification. Motion was also utilized in (Nguyen, Kuhnert, Thamke, Schlemper, & Kuhnert, 2012) to measure the resistance of vegetation pixels. (2) Approaches that perform object segmentation in a region merging process. In (Blas, Agrawal, Sundaresan, & Konolige, 2008), similar regions in road images were merged based on texton-based histogram profiles which were generated using K-means clustering on a fusion of L, a, b and pixel intensity differences, yielding 79% rate on classifying synthetic texture. In (Bosch, Muñoz, & Freixenet, 2007), the co-occurrence matrix was combined with RGB, HLS, and Lab to segment objects, and pixels were grown by minimizing a global energy function which integrates region and boundary information. In (Ligang Zhang, Verma, & Stockwell, 2016), roadside vegetation segmentation was achieved by progressively merging low confident superpixels to their most similar neighbors based on color and texture features. (3) Approaches that accomplish vegetation segmentation using prediction models. In (Zafarifar & de With, 2008), the pixel intensity difference was used in conjunction with a 3D Gaussian model of YUV for building a grass segmentation model, yielding 91% accuracy on 62 images. In (Schepelmann, Hudson, Merat, & Quinn, 2010), four statistic measures were clustered for segmenting illuminated grasses from artificial obstacles, achieving 95% accuracy on 40 region samples. In (Ligang Zhang, Verma, & Stockwell, 2015), opponent color intensity and color moments were used in conjunction with an ANN for vegetation classification. The local binary patterns and gray-level co-occurrence matrix features were combined with majority voting over three classifiers, including ANN, KNearest Neighbors (KNN) and Support Vector Machine (SVM), for dense vs. sparse vegetation classification in (Chowdhury, Verma, & Stockwell, 2015).
Visible features for object segmentation can also be extracted from remotely sensed data. In (Hu, Chen, Pan, & Hao, 2016), edges and initial object regions were extracted from satellite landscape images and used to find optimal objects by analysing the relationship between edges and regions in a region-growing process. In (Alshehhi, Marpu, Woon, & Mura, 2017), roads and buildings were segmented from satellite images using a patch-based Convolutional Neural Network (CNN). A set of shape features of adjacent regions was adopted to refine the segmentation results. Work (Cheng & Han, 2016) presented a survey of existing object segmentation (or detection) methods from remotely sensed images, and categorized them into four groups: template matching-based, knowledge-based, object-based, and machine learningbased. Although remotely sensed data in existing work are primarily aerial or satellite images, there have been increasingly more studies that have used drone-based image data (Malek, Bazi, Alajlan, AlHichri, & Melgani, 2014) and more convenient mobile laser scanning data (Li, et al., 2016).
Although promising results have been achieved, existing visible feature approaches still face the challenge of choosing or designing suitable visible features that are robust to the varieties in complicated real-life environments and scene content. Most evaluations are restricted to small evaluation data, and some studies just focus on artificial data which is far from the capacity of simulating real-world situations.
b) Invisible feature approaches utilize the reflectance properties of vegetation in the
invisible spectrum to recognize them. One of the most important features is Vegetation Index (VI), which indicates the differences between the spectral properties of vegetation and those of other objects on invisible spectrum wavelengths, such as green and near infrared. As an instance, a simple comparison between red and Near Infrared Ray (NIR) reflectance has shown high accuracy of detecting photosynthetic vegetation (Bradley, Unnikrishnan, & Bagnell, 2007). To enhance its robustness against environmental effects, the NIR was later extended to various versions, such as the Normalized Difference Vegetation Index (NDVI) (Bradley, et al., 2007), the Modification of NDVI (MNDVI) (Nguyen, Kuhnert, & Kuhnert, 2012b), and combination of NDVI and MNDVI (Nguyen, Kuhnert, Thamke, et al., 2012). To fully utilize merits of both visible and invisible features, they were also integrated for vegetation segmentation. In (Nguyen, Kuhnert, Jiang, Thamke, & Kuhnert, 2011), 3D scatter features extracted from LADAR data were fused with histograms of HSV to segment vegetation. In (Y. Kang, Yamaguchi, Naito, & Ninomiya, 2011), 20-D filter bank features extracted from L, a, b color and infrared channels were integrated in a hierarchical architecture for road object recognition. In (Nguyen, Kuhnert, & Kuhnert, 2012a) the opponent color and Gabor features were fused to calculate visual similarity between neighbouring pixels, which was used for growing vegetation pixels from initial seed pixels. The initial pixels were selected based on a fusion of NDVI and MNDVI features.
Compared with visible feature approaches, invisible feature approaches often have better robustness against changes of environmental conditions, particularly in extreme lighting conditions such as a dark environment. However, most existing invisible approaches still require specialized equipment to capture VI features and face the challenge of designing VIs reliable under complicated real-world conditions. Recent advances on using drone-based or ground-based sensors have greatly promoted the application of invisible features to supporting object segmentation from remotely sensed data.
2.2 Grass Biomass Estimation
Related work on grass biomass estimation can be roughly classified into three groups, including human inspection, remote sensing methods, and image processing algorithms.
a) Visual inspection is a common approach to estimate the vegetation biomass in real
practice, and it requires human going to the grass field and visually comparing the actual grasses with established criteria (e.g., ruler) to obtain the height of grasses. Although the results are generally accurate, human inspection is often labour-intensive, time-consuming, and costly. It also requires a certain degree of knowledge about field surveys and may need access permission from private landowners or relevant authorities. For field surveys in public roadsides as the case in this paper, access permission from private landowners is not required. However, permission from the government transport authority is still needed.
b) The vast majority of existing work on automatic biomass estimation are based on remote
sensing measurements. The remotely sensed data can be collected using satellite-based, airborne or terrestrial equipment such as optical spectral (Sibanda, et al., 2016), LIDAR (Vazirabad & Karslioglu, 2011), SAR (Santi, et al., 2017), and ultrasonic sensors (Moeckel, et al., 2017), (Chang, et al., 2017). VI is one of the earliest and most popular spectral measurements for biomass estimation. In (Payero, Neale, & Wright, 2004), 11 types of VIs were compared for predicting the height of grasses and alfalfa. The results showed that four of them have strong linear relationships with the plant height. Thus, it is recommended to select an appropriate type of VI for every particular type of vegetation. The LIDAR sensor is another popular way of remote sensing data collection and it has showed higher accuracy and more precise information about the canopy than ultrasonic sensor and VIs (Llorens, Gil, Llop, & Escolà, 2011). One popular LIDAR model in determining vegetation height is the Canopy Height Model (CHM) (StOnge, Hu, & Vega, 2008),(Grenzdörffer, 2014), which obtains the height by calculating the difference between the Digital Surface Model (DSM) and the Digital Terrain Model (DTM). However, the CHM requires the generation of both DSM and DTM. To remove the dependence on the DTM, Yamamoto et al. (Yamamoto, et al., 2011) proposed the use of a top surface model that is nearly parallel to the DTM. The model achieved accuracy close to one meter in measuring the mean tree height. For a summary of work on measuring plant height from LIDAR data, readers are referred to (Ahamed, Tian, Zhang, & Ting, 2011). Multi-frequency SAR data have also been long adopted for biomass estimation. In (Santi, et al., 2017), SAR data was used in conjunction with airborne LIDAR data to predict the forest biomass using an ANN predictor. Although these methods have achieved promising results, they are largely dependent on satellite or airplane platforms. Recent advances tend to employ more convenient and economical drone-based or ground-based sensors such as ultrasonic sensor (Moeckel, et al., 2017), (Chang, et al., 2017) and handheld mobile laser scanner (Ryding, et al., 2015). These platforms have greatly facilitated an easy and cheap deployment of various sensing equipment in both site-specific and large-scale field surveys, which has opened up new opportunities for a wide use of remote sensing techniques for biomass estimation. It is noted that features extracted from different sensors have also been combined in existing studies to provide more accurate estimation results, such as the fusion of VIs and terrestrial laser data (Tilly, Hoffmeister, et al., 2015), and fusion of ultrasonic and spectral sensor data (Moeckel, et al., 2017). For surveys of existing remote sensing methods for biomass estimation, readers are referred to (Lu, 2006), (Lu, et al., 2016), (Galidaki, et al., 2017).
c) Only few studies have investigated using image processing techniques for measuring the
height of plants from ground-based image data. In (Sritarapipat, et al., 2014), a method was presented for measuring the rice height from dynamically monitored figures of a rice field. The height is measured by matching the height of rice against the pre-known height of a bar that was initially installed in the field. Work (Juan & Xin-yuan, 2009), (Dianyuan & Chengduan, 2011) adopted a similar idea for obtaining the height of trees, which is calculated via a proportion transform of the coordinates of three pre-set makers on the tree. The extension to the system was presented in (Dianyuan, 2011), which used three marker points and a perspective transformation. Essentially, the principal idea of these approaches is to transfer the task of measuring the plant height to the task of locating pre-set markers using image processing algorithms. However, they have a strict requirement of field settings such as the height, location and angle of cameras, as well as a need of assistance to manually install reference markers. Similar to human inspection, these approaches are workable only for site-specific analysis and cannot be used for large-scale field analysis.
To combat with shortcomings in current work using ground-based image data, this paper introduces a novel DWCGP approach to automatically estimate the biomass of roadside grasses from image frames. The approach utilizes the connectivity of vertically oriented grass pixels along both vertical and horizontal directions to determine the biomass. It is a direct extension to our previous approach (Verma, et al., 2017) by weighting grass height with surrounding grass density to more effectively considering impact of both height and density. It is fully automatic and supports both small and large scale field tests. We further illustrate the effectiveness of the DWCGP approach in a practical task of fire risk identification on video data collected from a state road in Queensland, Australia.
This section describes the problem formulation of grass biomass estimation using image processing techniques, and then introduces the framework of the proposed DWCGP approach.
3.1 Motivations of the Proposed Approach
As reviewed in Section 2.2, there is no directly related approach that utilizes image processing techniques for estimating the biomass of roadside grasses from ground-based image data. To accomplish the goal of automatic estimation, we opt to follow and stimulate the traditional method of monitoring and calculating the fuel load of grasses (tonnes/ha) in field surveys, which often includes three steps: 1) collecting grass stems from a sampling region, 2) counting the number of grass stems, and 3) obtaining the total over-dry weight. The second step of counting the number of stems may not be necessary when only the final weight is needed. For the purpose of this paper, we follow studies on measuring crop biomass (C. Royo & Villegas, 2011), (Conxita Royo, Nazco, & Villegas, 2014), (Soriano, Villegas, Aranzana, del Moral, & Royo, 2016), which recorded and used the number of stems in the calculation of the biomass. The biomass equals to the product of average dry weight per plant and the number of plants per unit area. The three steps can be mathematically expressed using the following formula:
where, is the fuel load of grasses within the sampling region, is the number of grass stems in the region, indicates the length of the stem, and stands for the fuel load unit (e.g. fuel load per centimetre of grasses) for the stem. The fuel load is averaged over all stems.
For an image of roadside grasses, assume be the image window which corresponds to the sampling grass region selected in field surveys, and H and W indicate the
number of rows and columns respectively in . Let
column vector of , the target of automatic grass biomass estimation in is to find a projection solution that is capable of transferring properties of image column vectors in into an estimated fuel load :
Equation (3) enforces a constraint that the difference between the estimated fuel load and the physically quantified fuel load should be as minimal as possible.
To automatically estimate biomass from images in a similar way to manually measuring the physically quantified fuel load in field surveys, we make two assumptions: 1) grass stems can be analogously represented by columns of image pixels, and 2) the fuel load unit is equal for all stems of the same type of vegetation. Although the first assumption is not strictly correct in theory since grass stems do not always grow perfectly vertically in real-world situations, it is anticipated to represent an approximation of the vertical parts of stems and thus an indication of the grass height. Note that robust extraction of grass stems with different orientations is still a challenging task itself. The two assumptions greatly simplify the problem and provide a way to calculate the estimated fuel load in (2) based on column lengths of grass pixels in :
where, is the number of columns in , is the length of grass pixels in the column , and is a constant that enables a direct comparison between and fuel load in (1).
Equation (4) illustrates the basic concept of the proposed DWCGP approach that estimates the fuel load of grasses using image processing techniques in a way similar to the traditional method in field surveys. Specifically, the proposed approach calculates (4) in three consecutive steps, including measuring the length for the column, weighting the length by grass density in surrounding columns, and integrating the weighted lengths of all columns:
where, the is a mapping function from image pixels in a column to a scalar representing the length of the grass stem in the column, and is a mapping function from all weighted lengths to an estimated fuel load .
The proposed DWCGP is designed based on common knowledge in agriculture and forestry that the biomass yield is closely related with the plant height (Tilly, Aasen, & Bareth, 2015). However, our observation found that using solely the grass height will lead to incorrect biomass estimation in some practical conditions. If we take sparsely high grasses as an instance, we may get a relatively medium value (compared to sparsely low or densely high grasses) in the estimation of the grass height, but the actual density and biomass may be very low. This problem also exists in other similar conditions, such as low and high grasses co-existing in the sampling region. To solve the problem, the DWCGP also considers the grass density within a surrounding region relative to a specific column, which also has big impact on the grass biomass. This also agrees with the current practice of grassland curing in relevant government authorities (e.g. Country Fire Authority, Victoria, Australia: www.cfa.vic.gov.au/grass), which use both grass height and density as two major factors in human visual measurement of fuel loads of grasses.
To provide a solution that considers both grass height and density, the DWCGP calculates the connectivity of grass pixels along a vertical orientation in each image column and then weights it by the grass density in neighbouring columns. This is inspired by work (Rasmussen, 2004) which detected the primary direction of texture in a large neighbourhood by integrating the dominant texture orientation at image pixels. In (Verma, et al., 2017), we proposed an approach which takes the average length of all columns as an estimation of grass biomass. However, the approach relies on only height information along a vertical direction, but ignores density information along a horizontal direction. The proposed DWCGP also considers grass density in surrounding regions to provide more accurate prediction. For roadside grass images, it is observed that densely high grasses are normally characterised by long unbroken pixel connectivity across a certain number of continuous columns, while sparsely low grasses are often represented by short broken connectivity as shown in Fig. 1.
Fig. 1. Examples showing the differences in the classification results of grass pixels having a dominant vertical orientation between high dense grasses and low sparse grasses. Grass pixels with a vertical orientation are represented by a white color. Densely high grasses have longer connectivity along both vertical and horizontal orientations than sparely low grasses, which implies the importance of both height and density.
3.2 Framework of Proposed Approach
Fig. 2. Framework of the proposed DWCGP approach. For a sampling image window, the approach first classifies grass vs. non-grass pixels and then detects vertical vs. non-vertical dominant orientation at every pixel. The calculation of DWCGP is based on the connectivity of grass pixels along a vertical orientation in each column and the grass density in a surrounding region of the column.
Fig. 2 illustrates the systematic framework of the proposed DWCGP approach, which is composed of four main processing steps. The first step is to select a sampling grass window from a roadside image, which corresponds to the sampling grass region in field surveys, or can be any region of interest. The image window is used as the basic processing unit for biomass estimation. From the image window, grass region segmentation is performed to find the locations of all grass pixels using a feedforward ANN with color and texture features. As a parallel step, a Gabor filter vote process is employed to find the dominant texture orientation at each image pixel by performing a vote on the response magnitudes of multi-orientation and multi-scale Gabor filters on raw pixels in the grass window. Given the segmented grass pixels and their dominant orientations, a DWCGP algorithm is further presented to obtain the vertical connectivity of grass pixels in each image column and the grass density in a surrounding region, and then calculate average density weighted connectivity for all columns, which is taken as an estimated value of the biomass of grasses in the window.
3.3 Brown Grass Region Segmentation
Brown grass region segmentation aims to label every pixel in the sampling window into brown grass vs. non-brown grass categories. We mainly focus on brown grasses as they often present a much larger fire threat than green grasses in practice. To effectively represent the visual characteristics of grass pixels, it is critical to select a set of suitable features. This paper adopts color and texture features as both of them convey important information for representing vegetation and other roadside objects.
For color features, we adopt six color channels from the CIELab and RGB spaces. The Lab channels are well-known for their high perceptually consistency with human vision, while RGB channels may provide complementary information to Lab channels. It is observed that brown grasses are predominantly represented by a yellow color, while other roadside objects such as sky, road and tree are primarily characterised by non-yellow colors.
For texture features, we employ features extracted using the 17-D filter banks which were first proposed in (Winn, Criminisi, & Minka, 2005). The 17-D filter banks were designed to generate a universal visual dictionary for object representation and they have shown high accuracy of real-life object recognition. Studies (Yousun Kang, Kidono, Naito, & Ninomiya, 2008) also indicated that they outperform Leung and Malik, Schmid, and MR8 filter sets for road object segmentation. The 17-D filter banks include Gaussians with three scales (1, 2, 4) applied to L, a, and b channels, LoGs with four scales (1, 2, 4, 8) and the derivatives of Gaussians with two scales (2, 4) for each axis (x and y) on the L channel.
The final 23-D color and texture feature set can be expressed using:
where, represent color channels and
The extracted features can then be used for training a brown grass classifier. Amongst various types of classification algorithms, such as ANN, SVM, KNN and random forests, this paper chooses to use a three-layer feedforward ANN classifier due to its popularity and generalized capacity for classification. The ANN is composed of an input layer with 23 neurons, a hidden layer with N neurons and an output layer with one neuron. Given the extracted color and texture features , the input layer receives as input, performs multiplication and addition operations on based on a set of weights and constants, and finally transfers the calculated values using a linear or non-linear activation function:
where, is the transferred output for , and are trained weights and constants.
indicates the activation function, which is tangent sigmoid. The values are again taken as input of neurons in the hidden layer and similar operations as in (9) are performed from the hidden layer to the output layer. Finally, the output layer produces a probability using a linear activation function, and indicates the likelihood of brown grasses for each pixel at coordinates (i, j). The decision on the label of the pixel can then made based on :
where, represents a binary label for a pixel at (i, j), and is a threshold. The output forms the foundation of calculating the connectivity and density of grass pixels.
3.4 Vertical vs. Non-vertical Orientation Detection
To determine grass height using the proposed approach, it is a prerequisite to accurately find the dominant texture orientation at every image pixel. This is achieved by performing a vote on Gabor filter responses along multi-orientations to determine the strongest texture orientation at each pixel. The use of Gabor filter is inspired by its excellent capacity in capturing multi-scale and multi-orientation texture such as line and edge, which are common characteristics of grass stems in images.
For the sampling window , we first obtain its grey version by replacing each pixel by its average R, G and B values. Assume be 2D Gabor filter with an orientation and a scale , the responses of from S can be calculated by convolving the filter with intensities of all pixels in S:
Because the output is a complex value, we combine its real and imaginary components using a square norm, yielding a complex magnitude:
The can be used as an indicator of the absolute strength of Gabor filter responses. However, since the proposed approach is primarily dependent on orientation information, we remove the scale information by keeping only the maximum response of all scales for each orientation:
where, is the number of all scales.
Given a total of orientations, an orientation vector comprising of response magnitudes along all orientations can be obtained for every pixel at (i, j):
The dominant orientation of every pixel at (i, j) is then determined as the one having the maximum response among those of all orientations. This is essentially equivalent to performing a vote on response magnitudes along all orientations:
Because we consider only vertical vs. non-vertical orientations, is further converted into a binary category for a pixel at (i, j):
where, indicates the index of the vertical orientation, and .
3.5 DWCGP Calculation
Given the segmented grass vs. non-grass label and the detected vertical vs. non-vertical orientation for every image pixel, we further present an unsupervised DWCGP approach to predict the biomass in a sampling image window. The approach is composed of three major steps: a) determining the longest length of continuously connected grass pixels along a vertical orientation in every column; b) calculating the grass density in a surrounding region of the column, and c) obtaining density weighted lengths for each column and their average value over all columns. Thus, the DWCGP indicates an average of density weighted length of continuously connected grass pixels along a vertical orientation over all image columns and it is adopted as an estimation of the biomass.
Assume {grass; non-grass} be the segmented grass label and {vertical; non-vertical} be the detected dominant orientation for the pixel at the row and column in the sampling window , the value of can be converted into a binary value as follows:
The output often contains some isolated pixels that are either non-grass or non-vertical, i.e., , but are surrounded by vertical grass pixels. Those isolated pixels may severely impact the calculation of connectivity of grass pixels. To reduce the impact, those pixels are reassigned as vertical grass pixels as long as both its vertical neighbours are vertical grass pixels:
Note that boundary pixels in the region remain unchanged.
For all pixels , , in the column of , the lengths of continuously connected pixels can be calculated:
where,
to the existence of background noise and grass segmentation error, multiple lengths can be often obtained. To minimize the impact of those noise or error on the length results, only the longest length is kept and used as the length measurement for the column:
To take into account grass density, let assume be a surrounding region of the column and having a width of pixels as shown in Fig. 3. The grass density within can be obtained by taking the percentage of grass pixels:
where, indicates the number of grass pixels in and it can be calculated based on , i.e., .
Fig. 3. Illustration of the surrounding region of a current column where the percentage of grass pixels is calculated. The percentage provides contextual information about grass density around every column.
We then can obtain the density weighted length for column based on (20) and (21):
For all columns in , , we can get all their length measurements using (23):
The final DWCGP in can be obtained by taking the average of density weighted length measurements of all columns:
One problem arising is that the DWCGP in (24) and the physically quantified fuel load in (1) are calculated based on different measurement units. To make them directly comparable, one possible solution is to set a calibration factor a to the DWCGP:
Until now, we have been able to use DWCGP as an estimated value of biomass. It should be noted that (20), (22) and (24) correspond respectively to (5), (6) and (7) in the problem formulation.
The whole process of the proposed DWCGP approach is summarized in Algorithm 1. From the first to the last column in the sampling window, the approach begins by scanning the first pixel in every column and checking whether the pixel belongs to grass or non-grass (based on ANN classifier) and has a vertical or non-vertical orientation (based on Gabor filter votes). If the pixel is either non-grass or non-vertical, the approach continues moving to scan and check the next pixel in the column. Otherwise, if the pixel is grass and has a vertical orientation, it starts to count the number of continuously connected grass pixels having a vertical orientation along the same column. The counting continues until a non-grass or non-vertical orientation pixel is found, yielding multiple lengths of connected grass pixels for the column. However, we keep only the longest length in the column as it is a more robust indicator of the grass height than short lengths. For each column, we also calculate the percentage of grass pixels which is used as an indicator of the grass density of its surrounding region with a width of pixels. The longest length is then weighted with the density to take into account both grass height and density information. After obtaining the density weighted longest lengths of all columns, an ‘average’ operation is employed to obtain the DWCGP. This ‘average’ operation produces an average weighted height of all grasses in the sampling window.
In this section, we evaluate the proposed approach on estimating grass biomass and identifying fire-prone road regions. We also show the robustness of the proposed approach and compare its performance with existing approaches.
4.1 Experimental Data
There is no public dataset available that can be used for evaluating the performance of the proposed approach. We collected our own data from a total of 61 roadside sites (named F001 to F061) along the state roads within the Fitzroy region, Queensland, Australia. The ground truths of objective biomass were obtained by marking one square meter area in each of these sites using a quadrat, cutting and bagging the above-ground grass samples in the area, and then storing the samples in a heater (70 °C) for drying up for more than 72 hours, and finally weighting over-dry samples to calculate their fuel loads (tonnes/ha) using a standard formula.
Except for objective biomass, an image of 1936×1296 pixels was also captured for each site using a Dikon D80 camera. To facilitate direct comparisons of the estimated results with ground truth biomass, all images were taken by forcing the following constraints to camera settings: a) the lens directly faces roadside grasses, b) having the same height across all sites, and c) having the same distance from the camera to the sampling region for all sites. The strict settings for the camera are to ensure the accuracy of experimental results. In practice, it is nearly impossible to enforce the same settings to all road sites. Fortunately, the real-world test data for the proposed approach was collected by the DTMR using a vehicle-mounted camera, which was set to keep the same distance from the road boundary as much as possible. Thus, the camera has the same height for all road sites and we can assume that the camera has the same distances and angles to roadside grasses in the same relative locations across the captured images. For example, grass directly in front of the camera is at the same distance and angle for all images. For experiments in this paper, the image regions, which correspond exactly to the sampling regions for all sites, are manually cropped and used as the input to the proposed approach. To provide ground truths of grass density, all image regions are classified into one of three categories: sparse, moderate or dense, based on human visual observation. Samples of the three categories are shown in Fig. 4. The objective biomass and density categories of all images are listed in Table 1.
4.2 System Parameters
The ANN used for classifying grass vs. non-grass pixels has a structure of 23-N-2 neurons
where N is the number of hidden neurons and is set to 16 based on experimental comparisons. The ANN is trained using the Levenberg-Marquardt backpropagation algorithm with a goal error of 0.001 and a maximum epoch of 200. The training data comprises of 650 manually cropped grass and non-grass regions (e.g., tree, road, sky and soil), which are available at https://sites.google.com/site/cqucins/projects. The Gaussian filters for visual feature extraction have a kernel of 7×7 pixels, while the Gabor filters have a kernel size of 11×11 pixels, four orientations = (0o, 45o, 90o, and 135o) and five scales and . The width of the surrounding region of each column is set to be 5.
Fig. 4. Samples of dense, moderate and sparse grasses.
Table 1 Biomass Ground Truths and Estimated DWCGP for All Samples (Unit: Tonnes/Ha).
4.3 Performance of Brown Grass Segmentation
An important step in the proposed approach is to classify brown grass vs. non-brown grass pixels from roadside images. The classified regions are further used for calculating DWCGP and thus they may have big impact on the prediction results. This part shows the performance of brown grass segmentation. The evaluation dataset includes 50 frames that were randomly selected from the video data collected by the DTMR, Queensland, Australia. The frames cover the most common roadside objects such as brown grass, green grass, tree, soil, road, and sky, as well as various realistic environmental conditions. The pixel-wise ground truths of brown vs. non-brown grasses were manually annotated for all images. The category of non-brown grasses includes all objects other than brown grasses, such as tree, soil, road, and sky. Table 2 shows the classification accuracy obtained using the ANN classifier as compared to an SVM classifier with a RBF kernel. The same set of color and 17-D filter bank based features is used. It can be seen that using ANN slightly outperforms using SVM for both training and test data. Table 3 presents the confusion matrix using the ANN classifier. Brown grass pixels are easier for classification than other object pixels, which is expected as objects in the category of non-brown grasses may have big variations in their visual appearance and structure. Fig. 5 displays the segmentation results on several images, and we can observe that soil pixels are prone to be misclassified as brown grass pixels, likely due to their similar color.
Table 2 Classification Accuracy (%) of Brown Grass vs. Non-Brown Grass Pixels.
Table 3 Confusion Matrix (%) for Brown Grass vs. Non-Brown Grass Pixels Using an ANN Classifier.
Fig. 5. Results of brown grass (white pixel) and non-brown grass (black pixel) classification in sample images.
4.4 Performance of Biomass Estimation
The performance of estimating grass biomass is evaluated using Root-Mean-Square Error (RMSE) between DWCGP and objective biomass. The RMSE is a common measure for the performance of biomass estimation in existing work (Clark, Roberts, Ewel, & Clark, 2011),
(Anderson, et al., 2018) and it is calculated using
are calculated DWCGP and objective biomass respectively for the image and is the number of all images. The calibration factor a in (25) is the quotient of the total biomass divided by the total DWCGP of all samples. The unit of RMSE is the same as biomass unit (i.e., tonnes/ha).
(1) Performance vs. grass density. Table 1 lists estimated DWCGP and the corresponding
ground truth biomass for all image samples, as well as their mean values in three density categories. In overall, both DWCGP and biomass tend to have relatively low values for sparse grasses, relatively high values for dense grasses, and medium values for moderate grasses. However, there are also exceptions. For instance, the sample No 38 has comparatively high biomass and DWCGP although it is classified as sparse grasses based human visual observation. By contrast, the sample No 44 has relatively low biomass and DWCGP, but it is classified as dense grasses. The results suggest that, in the categorisation of grass density, even human eyes may make inconsistent predictions with ground truth biomass for individual samples. Thus, it is important to create a reasonably large number of samples to alleviate possible deviations in individual samples.
(2) Performance vs. non-vertical grass stems. In real-world situations, grass stems may
grow in various directions and the assumption of a vertical direction in the proposed approach may not be always true. Thus, we also report the results of the proposed approach using image samples with non-vertical grass stems, which are simulated by rotating images by a certain degree. Table 4 shows the RMSEs between DWCGP and biomass obtained using images rotated by [-10o, -5o, 0o, 5o, 10o]. As anticipated, the original images have slightly lower RMSEs than rotated images and the RMSEs tend to increase gradually along with an increased degree. The relatively small differences in RMSEs between original and rotated images is primarily contributed to the invariance of Gabor filter responses to small stem rotations and the adoption of a voting strategy over pre-defined orientations to determine the strongest texture orientation at each pixel. In this paper, Gabor filters categorize the texture responses into four pre-defined orientations (i.e., 0o, 45o, 90o and 135o) and find the dominant orientation at every pixel by performing a vote on all responses. Thus, texture with small deviations from its dominant orientation due to rotations would still be classified as having the same dominant orientation. For instance, grass stems with an orientation of 80o will be classified as with a vertical orientation because they are closer to 90o compared with other pre-defined orientations. The results confirm that the proposed approach is robust to grass stems that are marginally deviated from a vertical orientation.
Table 4 Performance of DWCGP Using Rotated Images (RMSE Unit: Tonnes/Ha).
(3) Performance vs. Gabor parameters. The DWCGP is designed based on a majority vote
over Gabor responses and thus its values may be significantly impacted by Gabor parameters. Table 5 compares the results of a maximum vs. an average operator for producing a single Gabor response for each orientation in (13); and a longest vs. a sum length of continuously connected grass pixels in (15). The two types of parameters directly determine the dominant orientation at each image pixel and DWCGP values respectively. We can observe that: 1) using a maximum Gabor response has slightly lower RMSEs than using an average response, and this probably due to a better capacity of using the maximum value in handling noise across different scales of Gabor filter responses. 2) The use of a longest length of grass pixels has produced lower RMSEs than the use of a sum length, which is expected because a sum length is prone to be affected by short lengths caused by noise in the environment and error in grass region segmentation and dominant orientation detection.
Table 5 Performance Comparisons of Maximum vs. average Gabor Responses across Scales, and Longest vs. Sum Lengths of Grass Pixels (RMSE Unit: Tonnes/Ha).
(4) Performance vs. width of surrounding regions. One key parameter of calculating grass
density is the width of surrounding regions for each image column, i.e., in (21). Table 6 shows the results obtained using different values of and the results indicate that using a width of 5 pixels seems to perform the best among the values tested. However, there are only small differences in RMSEs when different widths are used, and thus the width has limited impact on the results.
Table 6 RMSE Results vs. Width of Surrounding Regions (RMSE Unit: Tonnes/Ha).
(5) Proposed approach vs. previous approaches and human observation. Table 7 shows
comparisons of the proposed DWCGP approach with our previous VOCGP approach in (Verma, et al., 2017) and human observation. The VOCGP uses the same set of color and texture features and the same ANN classifier as the proposed DWCGP for brown grass segmentation. Different from DWCGP, it considers only the height of grass stems. The result of human observation is obtained by simply treating the biomass of each image as the mean biomass of its density category (i.e., dense, moderate or sparse). The proposed DWCGP approach shows a lower RMSE than the VOCGP approach and a RMSE close to human observation. The results indicate the possibility of using image processing techniques to achieve biomass estimation results as accurately as humans.
To show the significance of the performance of the proposed approach over benchmark approaches, we conduct statistical tests on the predicted results of all approaches using the twosample Kolmogorov-Smirnov (KS) test. The KS test is one of the most useful nonparametric methods for comparing the distributions of two data samples. In this paper, the null hypothesis is that the predicted biomass and ground truth biomass come from the same continuous distribution. Observed from Table 7, both the proposed DWCGP and the VOCGP have an H of zero, which indicates that both tests fail to reject the null hypothesis and thus the predicted biomass and ground truth biomass come from the same distribution. The DWCGP has a higher p value than the VOCGP, indicating its higher accuracy in predicting the biomass. As for human observation, an H equal to 1 and a p close to zero are obtained. This is expected because the predicted biomass of each image is simply taken as the mean biomass of its density category and thus there are less variations on the distribution of all samples.
Table 7 RMSE and KS Test Results of the Proposed DWCGP Approach Compared with the VOCGP Approach and Human Observation (RMSE Unit: Tonnes/Ha).
Note: the significance level for the KS test is set to be 0.05. If H=1, the null hypothesis is rejected. If H=0, the null hypothesis is not rejected. p indicates the probability of observing a test statistic as extreme as, or more extreme than, the observed value under the null hypothesis.
(6) Proposed approach vs. benchmark approaches. The current literature still lacks
approaches to grass biomass estimation that can be directly included for performance comparisons with the proposed approach. We compare the proposed approach with eight benchmark approaches, which use four texture feature descriptors and two kernel based prediction algorithms, as shown in Table 8. The feature descriptors are selected because they are the most widely used features to represent visual characteristics of objects in various computer vision tasks and have achieved state-of-the-art performance (Ahmed, Rasool, Afzal, & Siddiqi, 2017),(Soltanpour, Boufama, & Jonathan Wu, 2017). Similarly, the SVR and Kernel Ridge Regression (KRR) are also two popular prediction algorithms for regression (Huang, Han, & De la Torre, 2017). Thus, it is anticipated that they are qualified to be used as benchmark approaches to represent the state-of-the-art performance of existing machine learning solutions for automatic biomass prediction. The RMSEs of benchmark approaches are calculated based on five-fold cross validations. In each validation, sampling images from four folds are used for training and those from the rest fold for test. By comparing the results in Tables 7 and 8, we can see that all benchmark approaches have a similar performance of RMSEs around 6.5, except for the approach using Histogram of Oriented Gradients (HOGs) and KRR. The results represent the performance of using state-of-the-art algorithms for grass biomass prediction. The proposed approach has a lower RMSE than all benchmark approaches, showing the effectiveness and feasibility of the proposed concept that utilizes the density weighted connectivity of grass pixels along a vertical orientation to predict grass biomass. The proposed approach outperforms all benchmarked approaches in the KS tests, as the null hypothesis is rejected for all benchmarked approaches. The low performance of benchmark approaches maybe partially due to a limited number of training samples, as collecting large sampling data with objective biomass is time-consuming and effort-intensive. The proposed DWCGP does not suffer from this issue, as its calculation is based on unsupervised learning, which does not require the availability of training data.
Table 8 RMSE and KS Test Results of Eight Benchmark Approaches (RMSE Unit: Tonnes/Ha).
Note: SVR – Support Vector Regression, KRR – Kernel Ridge Regression, LBP – Local Binary Patterns, HOG -Histogram of Oriented Gradients, GLCM - Gray Level Co-occurrence Matrix. AlextNet indicates features learnt from the pre-trained AlexNet (Krizhevsky, Sutskever, & Hinton, 2012) on 61 sampling windows which are resized to 227×277 pixels. A RBF kernel is used for both SVR and KRR.
4.5 Performance of High vs. Low Fire Risk Classification
We also test the performance of low vs. high fire risk classification using the biomass predicted by the proposed approach. For this purpose, we manually selected a total of 382 frames (170 for high and 212 for low fire risk) from the video collected by the DTMR. To simulate real-world situations, no restriction was imposed on the selection process except that the selected frames should contain grass regions of low or high fire risk. In each frame, 15 overlapped sampling regions as shown in Fig. 6 were selected and manually annotated into a category of low, high, or unknown risk. Finally, we obtained 1,298 and 1,641 sampling regions for high and low fire risk respectively. Since the predicted biomass are continuous values, we use a threshold to classify them into a binary category of low or high risk. The threshold is set experimentally to 26.
Table 9 presents the confusion matrix of the classification results. An overall classification accuracy of 87.7% is obtained for all regions. It seems that high risk regions are slightly easier for classification than low risk regions using the proposed approach. Fig. 6 visually displays both good and bad classification results on sample images. In overall, the proposed approach accurately predicts the fire risk levels in most regions. However, as shown in the images in the bottom row, the performance also tends to be impacted by various factors, such as a large rotation degree of grass stems, an excessively dark color in brown grass regions, misclassification of brown grass pixels as soil, and similar texture structure between low and high grasses. These factors represent typical real-world challenging issues that should be paid special attention in further improvements to the proposed approach.
Table 9 Confusion Matrix (%) for Low vs. High Fire Risk Classification.
Fig. 6. Results of high vs. low fire risk classification using the proposed DWCGP approach in roadsie images. The top row shows images with good classification results, wherease the bottom row displays images with some misclassified regions.
4.6 Application to Fire-Prone Road Identification
One direct application of the proposed approach is to identify fire-prone roadside segments based on the estimated grass biomass. We evaluate the proposed approach on roadside video data collected by the DTMR, Queensland, Australia. A state road No. 16A in the Fitzroy region was chosen for the evaluation and a total of 100 frames were selected from 22 videos that were collected from this road. The frames cover the whole road with approximately the same distance of 200 meters between two frames.
Unlike field surveys where the sampling region is pre-known in every image, there is no information about the locations of sampling regions in the testing video frames. To provide an indication of the biomass in a whole frame, we choose 15 equal and overlapped sampling windows in each frame, as shown in Fig. 7, and obtain an average DWCGP over 15 windows. It is expected that, in images with dense and high grasses, most windows have high predicted DWCGPs and thus a high value of average DWCGP and a high possibility of fire risk. Fig. 8 shows the average DWCGPs of 100 frames that are displayed in an ascending order of their chainage on the road. Typical frames with locally high, low or medium DWCGPs are shown as well. The proposed approach using average DWCGPs achieves very encouraging results since comparisons between locally highest or lowest DWCGPs with the corresponding grass density levels indicate a high level of accuracy. The road segments that have frames of average DWCGPs above a certain threshold (i.e., 26) can be determined as fire-prone regions.
Fig. 7. Calculated DWCGPs of 15 grass windows in video frames collected from a state road No. 16A in Fitzroy, Queensland, Australia.
Fig. 8. Predicted average DWCGPs in video frames collected from a state road No. 16A. Typical frames corresponding to local maximum or minimum average DWCGPs are displayed. The frames are listed in an ascending order of their chainage on the road.
Automatically estimating roadside grass biomass from ground-based image data remains largely unexplored in current studies, but it plays a significant role in many practical applications. This paper presented a novel Density Weighted Connectivity of Grass Pixels (DWCGP) approach for the estimation of roadside grass biomass in ground-based images. We conducted extensive experiments on an image dataset to evaluate the effectiveness of the proposed approach in estimating grass biomass. The results show that, compared with the approach that does not consider grass density, the proposed DWCGP reduces RMSE from 5.84 to 5.52. It also has RMSE close to human observation and lower than eight baseline approaches, which use four popular texture features and two kernel regression algorithms. The approach shows good robustness to non-vertical grass stems and is little impacted by using different Gabor parameters and different widths of surrounding regions for calculating grass density. It also demonstrates encouraging results on automatically classifying low vs. high fire risk in image regions and identifying fire-prone road segments.
To further improve the performance of the proposed DWCGP approach in real-life applications, our results indicate that specific attention should be paid to situations such as large rotations of grass stems, and similarity in appearance between brown grasses and other objects. Possible solutions to these issues including incorporating more robust grass segmentation techniques, such as ensembles of prediction algorithms, and considering grass stems in multiple directions (e.g., 45 and 135 degrees) rather than purely a vertical direction during height calculation. Automatic biomass prediction using image processing techniques can be potentially impacted by data collection settings such as the angle and distance of the camera to the vegetation, and the relative location, size, and shape of sampling region within the region. For instance, the same image region captured using different angles or distances of camera may lead to different perception of the grass height, and they are likely to generate different results using the proposed approach. Thus, it is advisable to enforce the same camera settings for all field sites during data collection, however, this is still a difficult issue in real-world practice. It is also beneficial to collect more image data with ground truths of both objective biomass and grass height to facilitate the prediction using supervised learning algorithms.
Except for fire risk assessment, the proposed approach can also be potentially applied to other applications such as vegetation growth condition monitoring, effective vegetation management, and tree regrowth control. Being able to obtain site-specific parameters of roadside vegetation such as biomass, height, coverage and density can provide reliable and important indications of their current condition, growth stage and future tendency. Tacking the changes in these parameters is an effective way to improve effective vegetation management by detecting, quantifying, and handling the possible effects on the vegetation such as diseases, dryness, soil nutrients and water stress. Tree regrowth control aims to reduce road safety threats arising from roadside trees that grow progressively approaching the road boundary. Extending the proposed approach to automatically identifying parameters of these trees such as location, size, species, and distance to the road boundary can support the processes of determining suitable equipment to cut them and accurately predicting the associated cost.
This research was supported under Australian Research Council's Linkage Projects funding scheme (project number LP140100939).
Ahamed, T., Tian, L., Zhang, Y., & Ting, K. C. (2011). A review of remote sensing methods for biomass feedstock production. Biomass and Bioenergy, 35, 2455-2469.
Ahmed, M., Rasool, A. G., Afzal, H., & Siddiqi, I. (2017). Improving handwriting based gender classification using ensemble classifiers. Expert Systems with Applications, 85, 158-168.
Alshehhi, R., Marpu, P. R., Woon, W. L., & Mura, M. D. (2017). Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks. ISPRS Journal of Photogrammetry and Remote Sensing, 130, 139-149.
Anderson, K. E., Glenn, N. F., Spaete, L. P., Shinneman, D. J., Pilliod, D. S., Arkle, R. S., McIlroy, S. K., & Derryberry, D. R. (2018). Estimating vegetation biomass and cover across large plots in
shrub and grass dominated drylands using terrestrial lidar and machine learning. Ecological Indicators, 84, 793-802.
Andújar, D., Escolà, A., Rosell-Polo, J. R., Sanz, R., Rueda-Ayala, V., Fernández-Quintanilla, C., Ribeiro, A., & Dorado, J. (2016). A LiDAR-Based System to Assess Poplar Biomass. Gesunde Pflanzen, 68, 155-162.
Blas, M. R., Agrawal, M., Sundaresan, A., & Konolige, K. (2008). Fast color/texture segmentation for outdoor robots. In Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on (pp. 4078-4085).
Bosch, A., Muñoz, X., & Freixenet, J. (2007). Segmentation and description of natural outdoor scenes. Image and Vision Computing, 25, 727-740.
Bradley, D. M., Unnikrishnan, R., & Bagnell, J. (2007). Vegetation Detection for Driving in Complex Environments. In Robotics and Automation, IEEE International Conference on (pp. 503-508).
Campbell, N. W., Thomas, B. T., & Troscianko, T. (1997). Automatic Segmentation and Classification of Outdoor Images Using Neural Networks. International Journal of Neural Systems, 08, 137-144.
Chang, Y. K., Zaman, Q. U., Rehman, T. U., Farooque, A. A., Esau, T., & Jameel, M. W. (2017). A real-time ultrasonic system to measure wild blueberry plant height during harvesting. Biosystems Engineering, 157, 35-44.
Cheng, G., & Han, J. (2016). A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 117, 11-28.
Chowdhury, S., Verma, B., & Stockwell, D. (2015). A novel texture feature based multiple classifier technique for roadside vegetation classification. Expert Systems with Applications, 42, 5047-5055.
Clark, M. L., Roberts, D. A., Ewel, J. J., & Clark, D. B. (2011). Estimation of tropical rain forest aboveground biomass with small-footprint lidar and hyperspectral sensors. Remote Sensing of Environment, 115, 2931-2942.
Dianyuan, H. (2011). Tree height measurement based on image processing with 3-points correction. In Computer Science and Network Technology (ICCSNT), International Conference on (Vol. 4, pp. 2281-2284).
Dianyuan, H., & Chengduan, W. (2011). Tree height measurement based on image processing embedded in smart mobile phone. In Multimedia Technology (ICMT), International Conference on (pp. 3293-3296).
Fan, X., Kawamura, K., Xuan, T. D., Yuba, N., Lim, J., Yoshitoshi, R., Minh, T. N., Kurokawa, Y., & Obitsu, T. (2017). Lowcost visible and nearinfrared camera on an unmanned aerial vehicle for assessing the herbage biomass and leaf area index in an Italian ryegrass field. Grassland Science. (In press)
Fricke, T., Richter, F., & Wachendorf, M. (2011). Assessment of forage mass from grassland swards by height measurement using an ultrasonic sensor. Computers and Electronics in Agriculture, 79, 142-152.
Galidaki, G., Zianis, D., Gitas, I., Radoglou, K., Karathanassi, V., Tsakiri–Strati, M., Woodhouse, I., & Mallinis, G. (2017). Vegetation biomass estimation with remote sensing: focus on forest and other wooded land over the Mediterranean ecosystem. International Journal of Remote Sensing, 38, 1940-1966.
Grenzdörffer, G. (2014). Crop height determination with UAS point clouds. ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 1, 135-140.
Haibing, Z., Shirong, L., & Chaoliang, Z. (2014). Outdoor scene understanding using SEVI-BOVW model. In Neural Networks (IJCNN), International Joint Conference on (pp. 2986-2990).
Hamuda, E., Glavin, M., & Jones, E. (2016). A survey of image processing techniques for plant extraction and segmentation in the field. Computers and Electronics in Agriculture, 125, 184-199.
Harbas, I., & Subasic, M. (2014). Motion estimation aided detection of roadside vegetation. In Image and Signal Processing (CISP), 7th International Congress on (pp. 420-425).
Hu, Y., Chen, J., Pan, D., & Hao, Z. (2016). Edge-guided image object detection in multiscale segmentation for high-resolution remotely sensed imagery. IEEE Transactions on Geoscience and Remote Sensing, 54, 4702-4711.
Huang, D., Han, L., & De la Torre, F. (2017). Soft-Margin Mixture of Regressions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 6532-6540).
Juan, Z., & Xin-yuan, H. (2009). Measuring Method of Tree Height Based on Digital Image Processing Technology. In Information Science and Engineering (ICISE), 1st International Conference on (pp. 1327-1331).
Kachamba, D., Ørka, H., Gobakken, T., Eid, T., & Mwase, W. (2016). Biomass Estimation Using 3D Data from Unmanned Aerial Vehicle Imagery in a Tropical Woodland. Remote Sensing, 8, 968.
Kang, Y., Kidono, K., Naito, T., & Ninomiya, Y. (2008). Multiband image segmentation and object recognition using texture filter banks. In Pattern Recognition, 19th International Conference on (pp. 1-4).
Kang, Y., Yamaguchi, K., Naito, T., & Ninomiya, Y. (2011). Multiband Image Segmentation and Object Recognition for Understanding Road Scenes. Intelligent Transportation Systems, IEEE Transactions on, 12, 1423-1433.
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (pp. 1097-1105).
Li, L., Li, D., Zhu, H., & Li, Y. (2016). A dual growing method for the automatic extraction of individual trees from mobile laser scanning data. ISPRS Journal of Photogrammetry and Remote Sensing, 120, 37-52.
Llorens, J., Gil, E., Llop, J., & Escolà, A. (2011). Ultrasonic and LIDAR Sensors for Electronic Canopy Characterization in Vineyards: Advances to Improve Pesticide Application Methods. Sensors, 11, 2177-2194.
Lu, D. (2006). The potential and challenge of remote sensingbased biomass estimation. International Journal of Remote Sensing, 27, 1297-1328.
Lu, D., Chen, Q., Wang, G., Liu, L., Li, G., & Moran, E. (2016). A survey of remote sensing-based aboveground biomass estimation methods in forest ecosystems. International Journal of Digital Earth, 9, 63-105.
Malek, S., Bazi, Y., Alajlan, N., AlHichri, H., & Melgani, F. (2014). Efficient framework for palm tree detection in UAV images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7, 4692-4703.
Moeckel, T., Safari, H., Reddersen, B., Fricke, T., & Wachendorf, M. (2017). Fusion of Ultrasonic and Spectral Sensor Data for Improving the Estimation of Biomass in Grasslands with Heterogeneous Sward Structure. Remote Sensing, 9, 98.
Nguyen, D. V., Kuhnert, L., Jiang, T., Thamke, S., & Kuhnert, K. D. (2011). Vegetation detection for outdoor automobile guidance. In Industrial Technology (ICIT), IEEE International Conference on (pp. 358-364).
Nguyen, D. V., Kuhnert, L., & Kuhnert, K. D. (2012a). Spreading algorithm for efficient vegetation detection in cluttered outdoor environments. Robotics and Autonomous Systems, 60, 1498-1507.
Nguyen, D. V., Kuhnert, L., & Kuhnert, K. D. (2012b). Structure overview of vegetation detection. A novel approach for efficient vegetation detection using an active lighting system. Robotics and Autonomous Systems, 60, 498-508.
Nguyen, D. V., Kuhnert, L., Thamke, S., Schlemper, J., & Kuhnert, K. D. (2012). A novel approach for a double-check of passable vegetation detection in autonomous ground vehicles. In Intelligent Transportation Systems (ITSC), 15th International IEEE Conference on (pp. 230-236).
Payero, J., Neale, C., & Wright, J. (2004). Comparison of eleven vegetation indices for estimating plant height of alfalfa and grass. Applied Engineering in Agriculture, 20, 385-393.
Rasmussen, C. (2004). Grouping dominant orientations for ill-structured road following. In Computer Vision and Pattern Recognition (CVPR), IEEE Computer Society Conference on (Vol. 1, pp. 470-477).
Royo, C., Nazco, R., & Villegas, D. (2014). The climate of the zone of origin of Mediterranean durum wheat (Triticum durum Desf.) landraces affects their agronomic performance. Genetic Resources and Crop Evolution, 61, 1345-1358.
Royo, C., & Villegas, D. (2011). Field Measurements of Canopy Spectra for Biomass Assessment of Small-Grain Cereals, Biomass - Detection, Production and Usage: INTECH Open Access Publisher.
Ryding, J., Williams, E., Smith, M., & Eichhorn, M. (2015). Assessing Handheld Mobile Laser Scanners for Forest Surveys. Remote Sensing, 7, 1095.
Santi, E., Paloscia, S., Pettinato, S., Fontanelli, G., Mura, M., Zolli, C., Maselli, F., Chiesi, M., Bottai, L., & Chirici, G. (2017). The potential of multifrequency SAR images for estimating forest biomass in Mediterranean areas. Remote Sensing of Environment, 200, 63-73.
Schaefer, M., & Lamb, D. (2016). A Combination of Plant NDVI and LiDAR Measurements Improve the Estimation of Pasture Biomass in Tall Fescue (Festuca arundinacea var. Fletcher). Remote Sensing, 8, 109.
Schepelmann, A., Hudson, R. E., Merat, F. L., & Quinn, R. D. (2010). Visual segmentation of lawn grass for a mobile robotic lawnmower. In Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on (pp. 734-739).
Sibanda, M., Mutanga, O., & Rouget, M. (2016). Comparing the spectral settings of the new generation broad and narrow band sensors in estimating biomass of native grasses grown under different management practices. GIScience & Remote Sensing, 53, 614-633.
Soltanpour, S., Boufama, B., & Jonathan Wu, Q. M. (2017). A survey of local feature methods for 3D face recognition. Pattern Recognition, 72, 391-406.
Soriano, J. M., Villegas, D., Aranzana, M. J., del Moral, L. F. G., & Royo, C. (2016). Genetic structure of modern durum wheat cultivars and Mediterranean landraces matches with their agronomic performance. PloS one, 11, e0160983.
Sritarapipat, T., Rakwatin, P., & Kasetkasem, T. (2014). Automatic Rice Crop Height Measurement Using a Field Server and Digital Image Processing. Sensors, 14, 900-926.
StOnge, B., Hu, Y., & Vega, C. (2008). Mapping the height and aboveground biomass of a mixed forest using lidar and stereo Ikonos images. International Journal of Remote Sensing, 29, 1277-1294.
Tang, L., & Shao, G. (2015). Drone remote sensing for forestry research and practices. Journal of Forestry Research, 26, 791-797.
Tilly, N., Aasen, H., & Bareth, G. (2015). Fusion of Plant Height and Vegetation Indices for the Estimation of Barley Biomass. Remote Sensing, 7, 11449-11480.
Tilly, N., Hoffmeister, D., Cao, Q., Lenz-Wiedemann, V., Miao, Y., & Bareth, G. (2015). Transferability of Models for Estimating Paddy Rice Biomass from Spatial Plant Height Data. Agriculture, 5, 538-560.
Vazirabad, Y. F., & Karslioglu, M. O. (2011). Lidar for Biomass Estimation, Biomass - Detection, Production and Usage: INTECH Open Access Publisher.
Verma, B., Zhang, L., & Stockwell, D. (2017). Roadside Video Data Analysis: Deep Learning: Springer.
Winn, J., Criminisi, A., & Minka, T. (2005). Object categorization by learned universal visual dictionary. In Computer Vision (ICCV), Tenth IEEE International Conference on (Vol. 2, pp. 1800-1807).
Yamamoto, K., Takahashi, T., Miyachi, Y., Kondo, N., Morita, S., Nakao, M., Shibayama, T., Takaichi, Y., Tsuzuku, M., & Murate, N. (2011). Estimation of mean tree height using small-footprint airborne LiDAR without a digital terrain model. Journal of Forest Research, 16, 425-431.
Zafarifar, B., & de With, P. H. N. (2008). Grass Field Detection for TV Picture Quality Enhancement. In Consumer Electronics (ICCE), International Conference on (pp. 1-2).
Zhang, L., & Grift, T. E. (2012). A LIDAR-based crop height measurement system for Miscanthus giganteus. Computers and Electronics in Agriculture, 85, 70-76.
Zhang, L., Verma, B., & Stockwell, D. (2015). Roadside Vegetation Classification Using Color Intensity and Moments. In Natural Computation (ICNC), 11th International Conference on (pp. 1250-1255).
Zhang, L., Verma, B., & Stockwell, D. (2016). Spatial contextual superpixel model for natural roadside vegetation classification. Pattern Recognition, 60, 444-457.