Vision-based Detection of Power Distribution Lines

in Complex Remote Surroundings

Hrishikesh Sharma

, Rajeev Bhujade

, Adithya V

and Balamuralidhar P.

TCS Innovation Labs, Bangalore.

Email: hrishikesh.sharma@tcs.com, rajeev.b@tcs.com, adithya.v@tcs.com,balamurali.p@tcs.com
AbstractInspection and Maintenance of power lines is an
important and costly legal responsibility, mainly for public safety,
of a power distribution company. The traditional method of
manual, on-foot inspections have many disadvantages such as
long inspection cycles etc. Due to advances in sensors and
ight technology, an emerging cost-effective solution is to employ
Unmanned Aerial Vehicles (UAV) for the inspections. Automatic
detection and complete extraction of power lines from UAV-
based aerial imagery, having complex and varying natural
surroundings, is rst critical problem to be efciently solved for
enabling detection of line faults. In this paper, we propose a
solution based on a novel morphological operator, and a robust
image space heuristics for locating and complete extraction of
power lines. Our algorithm is a three-stage algorithm, and it
focuses on minimization of missed detection of line segments.
The entire algorithm was tested on a real outdoor video shot
for around 320 meters length of power grid using a xed-wing
UAV. No missed detection of important line segments across the
sequence of overhead video frames, and < 3% false positives
prove that our approach is very effective. We believe that our
algorithm can be easily ported for line detection problem in any
other real outdoor video as well.
Index TermsMorphological Detection; Heuristic Algorithm; Transmis-
sion Line Monitoring; UAV
I. INTRODUCTION
I
N any country, there are many installations and systems
which are located in outdoor areas. Some such systems
are vast, and their installation area may run into hundreds of
kilometers, e.g. a pipeline or a power grid. Some such systems
are inapproachable by humans, e.g. a dam on a river in a
tricky terrain, or power grid in thick forests. Maintenance,
both preventive and breakdown, of all such installations is
an obvious must, and typically a costly legal responsibility
towards public safety in most countries.
As is well-known, the current prevalent practice of peri-
odic human inspections for maintenance of such systems are
somewhat inefcient. Such inspection can sometimes lead to
some zones being inspected more frequently than needed,
and some not enough. Also, this task is difcult because
humans cannot enter the vicinity of high voltage. Semi-
automatic inspection using various non-human agents have
been proposed to overcome these inefciencies. The important
agents so far proposed are satellite, helicopter, crawling robots
and UAVs [1]. Inspection through helicopters is more efcient
than manual inspection, but it is quite expensive [2]. The
other option of hiring services of a remote sensing satellite
entails the problem that satellites have a xed schedule while
they move in their orbits [3]. Hence their visit time in a
particular zone may not suit the scheduling requirements of
an inspection, specially when there is an emergency. A third
option is of employing a robotic machine which can hang
and walk along the power cables, such as one by Hydro
Quebec Inc. that is being used by Indias National Power Grid
Corporation [4]. Generally, such robots are relatively heavy,
and are limited to survey of just vegetation growth in line
vicinity.
An emerging option out of all the non-manual methods is to
employ a UAV [5]. UAVs enable the in-depth reconnaissance
and surveillance of an area. A small UAV can acquire high-
resolution images that could be used in line inspections and
in fact, many other applications. By [6], the UAV option
incurs approximately
1
3
rd
the cost of employing helicopters
for inspection. Also, unlike robots, it is more reliable, and
covers detection of more faults. Finally, in the aftermath of
emergencies such as cyclone, earthquake, landslides etc., it is
quicker and easier to employ UAVs than to hire and employ a
helicopter for damage to the grid, or do a manual survey [7].
The transmission lines run into tens of kilometers, and
the image data acquired by UAV-mounted imaging sensor is
correspondingly huge. It is impossible for any human observer
to sift through all images/frames and locate a fault in real-time.
Hence vision-based automatic extraction of power lines and
fault detection along the extracted lines is a very important
problem to be solved for UAV-based monitoring. Towards
detection of power lines in overhead recorded video, especially
in varying complex natural outdoor background, we have a
designed a novel three-stage algorithm. We perform adaptive
thresholding to rst isolate the power lines in varying light
conditions along the line, and also due to time of day. We then
use a new erosion operator to select candidate set of edges
from which power lines have to be detected. The operator
retains the edges of a foreground region, especially straight
line segments, while eroding the interior. In the nal stage,
we use robust heuristics on pixel intensity to detect stripes
in image matrix that correspond to power lines. Overall, the
algorithm aims to minimize the missed detections, or false
negatives, since a minimum tracking of power line just below
the overhead camera is guaranteed in all frames. It also
additionally minimizes the mis-classication of random linear
feature as a power line segment i.e. a false positive, since long
linear stripes in image intensity is quite unique to power lines.
In fact, < 3% frames show residual false positives. 978-1-4799-2361-8/14/$31.00 c 2014 IEEE
In past, for line detection in visual band images, most of
the researchers have used Hough Transform at the core of
their detection algorithm [8], [9], [10], [11], [12]. It is well-
known that Hough Transform has computational complexity
of the order of O(n
3
) i.e. very high. Since we work with
video data of around 24 frames per second, faster processing
is needed. The core stage of our algorithm has a complexity of
just O(n
2
). Further, we have focused on detection and tracking
on video data with around 1000 frames to prove the robustness
and detection performance of algorithm, while most other
papers work with limited set of discrete images. In works fol-
lowing other approaches, [13] uses data from multiple sensors
mounted on UAV, including LiDAR, and focuses on somewhat
different problem of detection of intruding vegetation to the
transmission line corridor. [14] has a similar objective, and
uses photogrammetry to measure the distance between nearby
vegetation and the power line. A lengthy method for removal
of multi-region background is given in [15], but we achieve
the same using adaptive thresholding. Gabor lters have been
used for edge detection stage in [9], but [7] uses canny edge
detection for this stage which gives more accurate results.
Our approach is similar to approach in [16], but we differ
in the middle step in the sense that we use a simple erosion
operator. Further, we deal with overhead line detection. [17]
also uses image space heuristics to detect linear bands, but it
does that in very beginning which results in various types
of false positives which are then weeded out one-by-one,
resulting in long operations. Clustering in Hough space for
line detection has been used in [12], while we perform some
form of clustering in image/pixel space.
In the remaining paper, we rst describe our algorithm
in detail in section II. We then describe the nature of our
experiments in section III, which is followed by the main
section IV on results and analysis, before concluding this
paper.
II. PROPOSED DETECTION ALGORITHM
For overhead monitoring, the UAVs can y at a height
above the power-lines, and in relative proximity. The video
captured from this point will have the power-lines having
good brightness due to reection. However, the power-lines
in remote areas run long distances over complex terrain.
The image/video background hence can vary from trees and
patches of greenery to different at spaces such as barren
patches, along with other objects like houses and roads.
Accordingly, we have designed a new detection algorithm that
can detect lines in any such varied background.
Our detection algorithm consists of three stages as shown
in Figure 1. The rst two stages are related to preprocessing,
that are tuned to cater to the core line detection algorithm in
the nal, third stage. As explained earlier, the rst stage uses
adaptive thresholding to isolate the power lines in varying light
conditions. In the second stage, we then collect the signicant
edges of bright regions available after thresholding, as the
candidate edge set. For that, we use a new erosion operator,
which retains the edges of a foreground region, especially
Fig. 1: Functional Block Diagram of Detection Algorithm
straight line segments, while eroding the interior. In the nal
stage, we use robust heuristics on pixel intensity to detect
linear bands, or stripes in image matrix that correspond
to power lines. The choice of third stage is the key to
performance of our algorithm. As will be shown in details
in section IV, missed detections, or false negatives, are very
less since a minimum tracking of power line just below the
overhead camera is guaranteed. Similarly, mis-classication of
a random linear feature as a power line segment i.e. a false
positive is also very less, since long linear stripes in image
intensity is quite unique to power lines or any other linear
infrastructure in remote outdoor locations.
Algorithm 1 Algorithm for Detection of Power Lines
Perform a decomposition of the video into a sequence of
frames.
For Each Frame in the Sequence do
Adaptive Thresholding in a 33 neighborhood using
Gaussian Kernel
Majority logic based erosion of remaining background,
while retaining all linear features
Heuristic Detection of Power Lines in Image Space,
while simultaneously weeding out false positives
end for
A. Adaptive Thresholding
Isolating the power lines from a background that varies
in intensity becomes challenging with the use of a single
global threshold. Also, between frames the intensity of the
captured lines themselves vary according to the position of
the UAV. Finally, the inspection could be done at any time
in day i.e. different daylight brightness conditions, and the
detection algorithm must be able to detect successfully in
all conditions. An adaptive threshold algorithm that uses a
different threshold for every pixel in each frame according to
the pixel intensity in the neighborhood can address all these
issues. The algorithm employs a blur function to calculate
(a) Synthetic Test image with lines
3 pixels thick
(b) Eroded image after applying op-
erator with n=3 and threshold 2n
(c) Eroded image after applying op-
erator with n=5 and threshold 2n
(d) Eroded image after applying op-
erator with n=7 and threshold 2n
Fig. 2: Erosion Performance on Synthetic Image with Linear Features
the neighborhood intensity. The choice of the blur function
depends on the background characteristics, as well as its
customization/tuning. The power lines are just 2-3 pixels thick
in our case, and the background greenery, houses and roads are
spread across more pixels. Hence using a Gaussian kernel, that
calculates a weighted mean function in a 33 neighborhood
is able to suppress more background while retaining the
power lines when compared to any other kernel type or size.
This is because the kernel covers sufcient foreground and
background for a single pixel, and at times 2-3 pixel thick
lines.
B. Majority-based Morphological Erosion of Remaining
Background
The output of the adaptive thresholding algorithm is a binary
image with the power line pixel thickness retained from the
original image and certain bright background patches. The
perspective view of the power lines from the front facing
camera, during overhead tracking, results in lines of varying
thickness across the image. The lines are thickest near the
bottom edge of the frame and tapers off towards the horizon
near the top part of the image. Having a line of uniform
thickness makes it easier to distinguish it from the background
still present in the image after the previous stage.
To get such uniform thick lines, we use a novel majority-
based erosion operator. This operator is specic for line
detection, as it operates by thinning down the line thickness
to a single pixel. A nn all-ones matrix is as the kernel of the
operator is moved across the image. A sum-of-product (SOP)
of n or less was taken as a favorable candidate output and
the center pixel was set to one, else it was set to zero. The
condition here was that a line of any orientation cannot have
more than n as sum of product in a nn kernel. That is,
1) If the sum of products after imposing all-1 nn kernel
is > n pixels, then the foreground contains something
else than a power line. It should be suppressed.
2) If the SOP is = n, it has a power line crossing along any
rotation of the kernels principal diagonal, and nothing
else. It has to be retained.
3) If the SOP is < n, it has a power line crossing along
any rotation of the one of the kernels minor diagonals,
and nothing else. It has to be retained.
The morphological operator is used to retain lines in a binary
image and remove anything else. Basically it is a shape retainer
and in this application, the shape is a line. The interiors of
the foreground patches are removed and only their edges are
retained.
The size of the operator was chosen based on the distance of
separation between lines. It is obvious that for a nn kernel,
the distance between the lines should be a minimum of n
pixels, for both the lines to be retained. The performance of
this operator for a synthetic test image, a real sideways power
grid image, and a real overhead power grid image is shown
in Figs. 2, 3 and 4. The images demonstrate that a kernel of
size 33 reduces false negatives the most, while a kernel of
size 77 reduces false positives the most i.e. a tradeoff. The
images also demonstrate that the erosion operator works irre-
spective of the orientation of the power lines. To understand
detailed parametric tuning of this operator, refer to [18].
C. Heuristics based line detection
It is a known fact that metallic objects reect majority of
electromagnetic radiation in visual band that impinges upon
it. Hence it is expected that other than visible sky above the
horizon in a typical overhead frame, no other artefact within
the frame will have high intensity, as captured within the pixel
intensity matrix. Since we are able to erode sky easily due
to usage of our erosion operator, power lines remain as the
most luminant artefact in the post-erosion frames. The same
has also been observed experimentally, as a predicate at all
stages of processing: from colored images to HSV-represented
images to binary images, post erosion. Specically, in an
HSV representation of image, power lines consistently showed
intensity > 65% in the value dimension, along the surface
of (thin) power line captured. The power lines captured in
our video are 3-pixel wide. However, in the perspective view
captured by the front facing camera, the parallel edges of
power lines appear to meet at the horizon. Hence the line
thickness slowly tapers off as its distance from the camera
increases.
The above property holds true after erosion also. Post ero-
sion, the lines are single-pixel wide and also have an expected
slightly staggered appearance due to overhead detection and
discreteness of image capturing grid on the CCD backplane
of the camera. The minor stagger, a uniform tilt of 1 pixel to
(a) Power Grid Sideway Image (b) Eroded image after applying op-
erator with n=3 and threshold n
(c) Eroded image after applying op-
erator with n=5 and threshold n
(d) Eroded image after applying op-
erator with n=7 and threshold n
Fig. 3: Erosion Performance on Grid Sideways Image
(a) Power Grid Overhead Image (b) Eroded image after applying op-
erator with n=3 and threshold n
(c) Eroded image after applying op-
erator with n=5 and threshold n
(d) Eroded image after applying op-
erator with n=7 and threshold n
Fig. 4: Erosion Performance on Grid Overhead Image
right or to the left, manifests as small vertical line segments
that are stacked one above the other in a staircase manner. The
more vertical the line during overhead detection, the bigger are
the length of the line segments, and lesser it staggers.
We use this consistent heuristics, which has been also used
in [16], [17] in a different form, to trace the lines. In overhead
detection, power lines typically run from bottom to top of the
image matrix. To trace the lines, rst, their starting locations
have to be identied. The front facing camera has the closest
view of the line at the bottom of the image. Hence the bottom
portion of the image has the lines in greatest prominence
in all frames, with no breakages in between. In the bottom
few rows of the post-erosion matrix, we look out for vertical
runs of all 1s. We also assume that lines are separated by
at least 3 pixel distance, which can be tuned, and hence
use windowing to locally identify starting positions. Once
the starting positions are identied, row-wise scanning from
bottom to top is continued to greedily detect and accumulate
the line length, till a break in the line is encountered.
III. EXPERIMENTS AND DATA COLLECTION
The image capturing can be done using many cameras e.g.
visual or near infrared camera. The popular imaging is done
by using visual camera. A video was captured as the visual
band image data sequence that would be used for evaluation of
our algorithm. A 320 meter section of line in the outskirts of
Bangalore was chosen as the test site. A xed-wing mini-UAV
was own at a speed of around 35 km/hour. The positioning of
the UAV was done so that it could ow overhead to the power
grid. This implied that the camera tilt was also front-facing.
This minimized the amount of occlusion among power lines.
The camera captured the frames of (standard) size 640480
pixels, at a frame rate of 24 fps. The UAV ew over lines
between two poles separated by one intermediate pole. The
overall video consisted of 782 frames, and lasted around 32
seconds. The background of the power lines was varying, and
typically consisted of vegetation, sky, few unpaved roads and
houses. For lack of space, while demonstrating the results, we
only show few sample frames out of this video.
IV. RESULTS AND PERFORMANCE ANALYSIS
The algorithm was implemented using OpenCV software
package. While the algorithm is tested over around 800 frames,
to showcase our algorithms effectiveness, we show tracking of
power line in two specic frames here. The two images have
been chosen with different enough background. They were
chosen in order to show that the proposed approach works
well in different conditions. The background of gure 6a is
more complicated than the one of gure 5a. The images after
complete processing and tracking are shown in Figs. 5b and
6b, respectively. The results show qualitatively that we are
able to detect majority of overhead power lines. For overhead
monitoring, this is very useful result as we quantitatively show
next. For fault detection which is the next stage in power
line monitoring, this initial success of power line detection
is important.
For quantitative analysis of results in form of tracked line
length, we present both the worst case and average case
analysis of our algorithm. Before presenting the analysis, let
us discuss a lower bound on the required length of line to
be tracked between two successive frame captures, in units of
pixels. The UAV was own at a speed of approximately 10
(a) Sample Video Frame 1 (b) Frame Overlaid with Detected Power Lines
Fig. 5: Detection of Lines in Sample Frame 1
(a) Sample Video Frame 2 (b) Frame Overlaid with Detected Power Lines
Fig. 6: Detection of Lines in Sample Frame 2
m/s, and the frame capture rate was 24 fps. Hence between two
frame captures, UAV ies over around 0.4 meters. Though not
shown for brevity, it is easy to calculate that for two poles that
are 100 meters apart as per the grid design, the wire between
them covers 3/4 of image length i.e. 360 pixels. Hence 0.4
meter corresponds to 1.44 pixel distance. The inter-frame
distance poses a very important requirement on line tracking.
This is because in overhead detection, the detection of line
segment just below the front-facing camera is very good.
Hence if we can somehow always detect power line segment
equal in length to the distance covered by UAV between two
successive images from the bottom of each image, for all
such successive pairs, then across the length of video, we can
successfully claim that we have tracked the entire power line.
For statistical analysis, we chose 281 sample frames, which
are placed uniformly 3 frames apart in the real video sequence.
From analysis of detected length data for all these images
frames, it was seen that only 9/281 frames had few false
positives. For all other frames, it was again seen that the
minimum length of line detected is around 17 pixels from
bottom of the frame. The power line in each frame is around
300 pixels long. However, when we compare this to our
important requirement of being able to track at least 1.44 pixels
from the bottom of each frame, then we have denitely done
around 10 better, even in worst-case.
Moving to average case analysis, we did a classication of
long and short lines in each frame, by thresholding on 10%
variation around the mean. It was found that only 13% of the
lines have no. of detected long lines lesser than no. of detected
short lines. This implies that in majority of frames, lines were
detected that were long enough. Further, we also took a global
mean of all such detected lines across all frames. A global
average of 179 pixels, out of 300 pixel long representation of
a true line in our video, is not only 125 better than required
bound, but also covers 60% of the overall length of the visible
grid till the horizon.
V. CONCLUSION
In this paper, we have provided an algorithm of detection
of power lines in complex and varying outdoor surroundings.
Automatic detection of power lines is an important rst step
in usage of increasingly popular solution of using UAV as
a sensing platform for many applications, including power
grid monitoring. The design of algorithm is based on a novel
morphological erosion operator, and a robust image pixel space
heuristics to extract power lines. The entire algorithm was
tested on a real outdoor video shot for around 320 meters
length of power grid using a xed-wing UAV. In all frames,
we are able to detect at least few lines for which detected
initial length is minimum of 17 pixels, more than the minimum
required length of 1.44 pixels for overhead detection in video
monitoring. In fact, in 87% of the frames, long detected lines
are in majority and cover upto 60% visible line length till
horizon. Thus, we have 0% missed detection when it comes
to important line segments. Also, false positives are far and
few, and were found to be limited in only 9/281 analyzed
frames. We are currently working on further improving the
degree of detected line length in each frame on two counts.
Whenever there is a bright barren patch beneath a power line,
the greedy line extraction in last stage is found to stray into
the patch. Also, in cases when two lines at different altitude
nearly align in the same plane in which UAV is ying, the
captured image renders them very close, typically 1-2 pixels
apart. We are working on detecting all such closely rendered
lines as well. Both of them require ne-tuning of one stage
respectively, and not any redesign of algorithm. Hence, and
overall, we believe that our algorithm for power line detection
is robust enough with good performance, and hence can be
also applicable to any line detection problem in any other real
outdoor video as well.
ACKNOWLEDGMENTS
We thank Prof. Omkar and his research group from Dept.
of Aerospace, Indian Institute of Science, Bangalore for col-
laborating and providing us with test video data to run our
algorithm.
REFERENCES
[1] D. Jones, Power Line Inspection a UAV Concept, IEE Forum on
Autonomous Systems, 2005.
[2] X.-f. Zhang, G. Chen, W. Long, Z.-f. Cheng, and K. Zhang, Current
Status and Prospects of Helicopter Power Line Inspection Tour with
LIDAR, Electric Power Construction, vol. 3, p. 013, 2008.
[3] F. N emec, O. Santolk, M. Parrot, and J. Bortnik, Power Line Harmonic
Radiation observed by Satellite: Properties and Propagation through the
Ionosphere, Journal of Geophysical Research, vol. 113, no. A8, 2008.
[4] Indias Power Grid Corp. to deploy robotics
technology, http://www.elp.com/articles/2011/11/
india-s-power-grid-corp-to-deploy-robotics-technology.html, 2011.
[5] S. Adams and C. Friedland, A Survey of Unmanned Aerial Vehicle
(UAV) usage for Imagery Collection in Disaster Research and Man-
agement, in International Workshop on Remote Sensing for Disaster
Response, 2011.
[6] T. Skrzypietz, Unmanned Aircraft Systems for Civilian Missions:
Policy Paper, Brandenburg Institute for Society and Security, Tech.
Rep., 2012.
[7] Z. Li, Y. Liu, R. Hayward, J. Zhang, and J. Cai, Knowledge-based
Power Line Detection for UAV Surveillance and Inspection Systems, in
IEEE International Conference on Image and Vision Computing, 2008,
pp. 16.
[8] T. W. Yang, H. Yin, Q. Q. Ruan, J. D. Han, J. T. Qi, Q. Yong, Z. T.
Wang, and Z. Q. Sun, Overhead Power Line Detection from UAV
Video Images, in IEEE International Conference on Mechatronics and
Machine Vision in Practice, 2012, pp. 7479.
[9] C. Mu, J. Yu, Y. Feng, and J. Cai, Power Lines Extraction from
Aerial Images based on Gabor Filter, in International Symposium on
Spatial Analysis, Spatial-temporal Data Modeling, and Data Mining.
International Society for Optics and Photonics, 2009.
[10] G. Yan, C. Li, G. Zhou, W. Zhang, and X. Li, Automatic extraction of
power lines from aerial images, IEEE Geoscience and Remote Sensing
Letters, vol. 4, no. 3, pp. 387391, 2007.
[11] W.-G. Tong, B.-S. Li, J.-S. Yuan, and S.-T. Zhao, Transmission Line
Extraction and Recognition from Natural Complex Background, in
IEEE International Conference on Machine Learning and Cybernetics,
vol. 4, 2009, pp. 24732477.
[12] J. Zhang, L. Liu, B. Wang, X. Chen, Q. Wang, and T. Zheng, High
Speed Automatic Power Line Detection and Tracking for a UAV-Based
Inspection, in IEEE International Conference on Industrial Control and
Electronics Engineering, 2012, pp. 266269.
[13] S. J. Mills, M. P. Castro, Z. Li, J. Cai, R. Hayward, L. Mejias, and
R. A. Walker, Evaluation of Aerial Remote Sensing Techniques for
Vegetation Management in Power-line Corridors, IEEE Transactions on
Geoscience and Remote Sensing, vol. 48, no. 9, pp. 33793390, 2010.
[14] W. Zhang, G. Yan, N. Wang, Q. Li, and W. Zhao, Automatic Block
Generation and 3D Line Extraction in Photogrammetric Power Line
Inspection, in IEEE International Geoscience and Remote Sensing
Symposium, 2007, pp. 29202922.
[15] P. Mazurek and K. Okarma, Application of Background Estimation and
Removal Techniques for the Extraction of the Power Line Components
on the Digital Images for the Automatic Power Line Inspection Sys-
tems, Pomiary, Automatyka, Kontrola, vol. 54, pp. 698699, 2008.
[16] Z. Xiao, Study on Methods to extract Transmission Line Informa-
tion from High-resolution Imagery, in International Symposium on
Multispectral Image Processing and Pattern Recognition. International
Society for Optics and Photonics, 2009.
[17] L. Yuee and L. Mejias, Real-time Power Line Extraction from Un-
manned Aerial System Video Images, in 2nd International Conference
on Applied Robotics for the Power Industry (CARPI), 2012, pp. 5257.
[18] R. Bhujade, A. V, and B. P., Design of a Novel Morphological Erosion
Operator specic to Power Line Detection Problem, Innovation Labs,
Tata Consultancy Services Ltd., Tech. Rep., 2013.