Conservation of Information in Search: Measuring the Cost of Success

William A. Dembski and Robert J. Marks II
IEEE Transactions on Systems, Man and Cybernetics A, Systems & Humans, vol.5, #5, September 2009, pp.1051-1061 Cite as: William A. Dembski and Robert J. Marks II "Conservation of Information in Search: Measuring the Cost of Success" IEEE Transactions on Systems, Man and Cybernetics A, Systems & Humans, vol.5, #5, September 2009, pp.1051-1061

Abstract

Conservation of information theorems indicate that any search algorithm performs on average as well as random search without replacement unless it takes advantage of problem-specific information about the search target or the search-space structure. Combinatorics shows that even a moderately sized search requires problem-specific information to be successful. Three measures to characterize the information required for successful search are (1) endogenous information, which measures the difficulty of finding a target using random search; (2) exogenous information, which measures the difficulty that remains in finding a target once a search takes advantage of problem-specific information; and (3) active information, which, as the difference between endogenous and exogenous information, measures the contribution of problem-specific information for successfully finding a target. This paper develops a methodology based on these information measures to gauge the effectiveness with which problem-specific information facilitates successful search. It then applies this methodology to various search tools widely used in evolutionary search.

[ PDF | IEEE ]

Errata

  • June 16, 2010

    Mathematical Errors

    GIF89a-PPP ```@@@pppа000!,- dihl($x|΁X}Ȥr): PP8z[ H` 5 clz $G OnL% |& $"X~} JmX'Z#q#>L"s  + |A## l=Ke%(z 8JO $s)og1HfgOހn)d#JlpGh2Y u@ G  bQ,L0$ ZM*<?Ѹ+̈́R(&`0N+ XC`pZSp'AY| u0X@R=L 8/0 Of͡@_Y#0kR#89U P:€}e]KY~4H f@^͐D֮5YʁwwNjD<5vt# @EЃ9v5,#߁lUѽvD%L@8!?4B(,0(c&lr AÎ<&0 Tzl ؄B }OcP h"7%d-A)Ng+ %8% V)i=QNc !ec#S@oc4 @6@ iQ>_ { d IƨPJaƈ7v|!*j( kY" eVWRl^&@gqGLf~h9ŭj TƝhPWՑ)JÍ*~C:LH9^cZ_+2E mzFM"'zl1HR WzkB.96[p)T=̉V=Cp*$FWMg2'YT5.} 4f$@vFg))[\_ +zJFݬ8h@D35$m oS1W |OI# ($h( MF`eS/đMOMtRc 7Jv3By18YM WA}S@d' w 3 D["0 H6Y(VO$(^ddx`Td>(_c8n1dY.d9U3vXb#Ilqd_%V}5Y6{0sv)l9J :vw\r-V~x4^o%8[s'nQ+;GIF89a,```PPP 000@@@ppp!,, dihl;B.tm%S`1jx䊑xd$A@)Uz -`莧H(8q-8435(Dr3 # k#dIf"&L, b% uz%)) " #O"0 (m)& V '(g" d0 xc# ]m;@x#>~GWNpcCf# |jb- @}VL%J IG"$gRHu,ȁҴ@ MYCB{~SD2BFPkM4! ˖{4.+ 0;X:sbY@fV$ȹ:1s S*pP,Y mG"xh)E&s3nD$H-"ËOp fK`uO}6@Iu@t"e@YH Xf H\D@FŇHAzT„ܕ JraXT#O*cC >*ѓ $4Q (Mڀ"%Z&I> E)+'d ЉC/21ȰV&pr[h*6|JLE6N<(ĥ-jcFjBX@Vel 8i=D)u p6q4ӟ6 KrdK\2eXfNR5VUYUnS$pe GyWG V%pE["Y*GD^bp:9({adh!X iI;GIF89a,```PPP 000@@@ppp!,, dihl뎐tm88ɔP52H<3 d N.`c ej;! 'vxN9dN~& (EuJ # m#}=h"&M8 d%" xj&)5  #P"1 (o). W '((i" f1 {e# ^+s.U 4(i,i#Ɛ01I/1@DARP1 b )V@Cۈf%ʡÇQW̵\ SxJu(Q!8r`>L- "!{K9A0*!*i(]aJ"R|, KT mzD1 \ ,/+RR܎kZ*_|xIx^d3JhU5f|j EA:Oӫ_Ͼ>A^ \yd' ( 7`pPh sx1}Wl ЀJp ְVoGŊKMQ}Z&Ѕ ӐrQLI>&(Qy@WyYrhi")i * `f `(nF 0P ,3F6u]BB4Di%#΂{*a胖jg5S=qzP=`$`x)>z %ӬFi4RI]tݝQi+EkN:1S K=TId0 mUi]V]} L@dzxL*5Z0 vWKϪ@`*FaU) qdɭBf&@5[;GIF89a+PPP ```а@@@000ppp!,+ dih1tmxuapH,Oql:IZ"B@z9L.Kɳz}NQ &.1{$i#n qx&0or./cB  B y&>G A % F 3#G ;# /C"Ŀ*%9%E#J4ut9 $; C#3$}8 -(q#A4L-YEX@4A$H!|njD4c: 7;GIF89a+PPP ```а@@@000ppp!,+ dih1tmx|YpH,,l:OAZ! z".3%z>Kܺ~A|iK#%o yq$0p./dH  B s&?X = % F ,"jY ;# /C")ʍS9%E#3vu9 $&; C#3$XiAm[ [@qADVۘe!UD#I"?n$`OBG0—p,PGJJ$ͤ6HS2" ` &nr/2xFx&}6ݻx˷_ E0p ʅ76kǐ#k2@rM<8b]5jӪ1L]5pkN l@gWw ;06"Jd4^\Lx<;#5>i<(<Ow G$֠rY`P]A Q4.9&Gy% |W`@xC|">#Eka覂T@-I.P# - B T#X P\Q4%cop4h \@N:tsBShXĥlfx>fJoe@Q!mA$<]$Bcs&j E>34 N"$O(ZfS=Wq֭!'4֬"%a!2 b ob% [ v^\gй9X!Rx*_Y|[_gx( "^oD|h#T!'"iVv,GZְ>ۏ'|?m@ 0OU9C WMwi`! ~%ET;GIF89aV+PPP ```а@@@000ppp!,V+ dih1tmq|oH r"dhPKt:B wԊ1InB i" ;zLGywO $b#QN jH ]0}X  ./,9&d%`ck&hmu= $4 [Y= q@6 $ ۽<$  O& )2 * H0Y 0xā4T/LyL `"4" `#iɦ1'`@+lz50RIi VRT ~PEL*AeA @eb H3EB'8Ag# tܶwN,4 pw̒k"OU4,vao m|% VF<5`k=bul}7cI>CfL @e{VWma,Y%2 m׫B@9+V > g?0 ;GIF89adPPP ```а@@@000ppp!,d C9blp,mas.C U@!1< KQ .e-"ME%˶EW8)-~"" )`w{+k] ++ ) -""f#-X| <"w)o,ƥ#ч @Z}" K sV+*8y\+ ܻع' * &M JG 2kg!OFɳ&O ]4aU[q=R& I@F)h ʉ XKfE|tb+ sα,@W(1 + mb mD+ax^:%dK h*ͦfȖl%rU}^H@o#qŃnУGWҳkAŽ;GIF89amPPP ```а@@@000ppp!,m C9blp,4G[9 BxԑR8Xˤ e@5!d$J:&D^ qV (`` )-h"* q* +e` r- ) "")+Y+[` ="}"YqҲnܿ AΝ4S I ,r`ӏM(Ao_p V&&yJA-0\XZQz직S"K7 (@j `P" D'E0imL[9UWEtQ$+R ;*=}rm+FYr *WY.@p V-#X; ^ XC6 gUTcj|ZalUm@lǞ )8PoA0$@'r: ԣ[ (N PxIa"B;

    Thanks toDietmar Eben for pointing out these errors.

    p 1056, 2nd col, line 12 (eq. 27):

    I_+ = - \log (\frac{1 - F^{\Phi}_{\Delta \kappa} (0)}{1 - F_{\Delta \kappa} (0)})
    should read
    I_+ = \log (\frac{1 - F^{\Phi}_{\Delta \kappa} (0)}{1 - F_{\Delta \kappa} (0)})

    p 1057, 1st col, line 5:

    Q  \approx \frac{2 \log(L)}{\log(1-2\mu)}\,
    should read
    Q  \approx - \frac{2 \log(L)}{\log(1-2\mu)}\,

    p 1057, 1st col, line 7:

    I_{\oplus} \approx \frac{L}{\log(L)} \log(1-2\mu)\,
    should read
    I_{\oplus} \approx \frac{L}{2 \log(L)} \log(1-2\mu)\,

    p 1057, 1st col, line 9:

    I_{\oplus} \approx \frac{2 \mu L}{\ln(L)}\,
    should read
    I_{\oplus} \approx \frac{\mu L}{\ln(L)}\,

    p 1057, 1st col, line 10:

    I_{\oplus} \approx 0.0022 b\,
    should read
    I_{\oplus} \approx 0.00109 b

  • May 14, 2010

    Weasel Interpretation

    In the paperConservation of Information in Search: Measuring the Cost of Successwe mistakingly referred to Dawkins'Weaselas apartitioned search. This interpretation of Dawkins's simulation was, however, wrong. Concerning generations, Dawkins writes The computer examines the 'progeny' of the original phrases, and chooses the one, however slightly, most resembles the target phrase... Since phrases is plural, Dawkins uses more than one child per generation. Thus, our initial interpretation was incorrect. See further discussion on theWeasel Wareresearch tool page.

Related Resources

The Evolutionary Informatics Lab
XHTML CSS